Semantic information assisted task-oriented image compression method, system and equipment

Abstract

The invention relates to the technical field of image compression, and discloses a semantic information-assisted task-oriented image compression method, system and device in order to solve the problems that the data volume of images to be transmitted is large and the airborne communication bandwidth is limited in unmanned aerial vehicle communication. The method comprises the steps that semantic segmentation is conducted on an original image according to a task knowledge base, a plurality of task related information units and background information units are obtained, and the task knowledge base comprises task backgrounds, task names, task requirements and task correlation scores; calculating the weight of each task related information unit according to the task correlation score and the data size of each task related information unit; according to the weight of each task-related information unit, bandwidth allocation is carried out on each task-related information unit and the background information unit; and according to a bandwidth allocation result, performing image compression on each task related information unit and the background information unit to obtain a multi-scale compressed image. By adopting the method, high-quality image transmission can be realized under the condition that the communication bandwidth is limited.

Description

Technical Field

[0001] The present application relates to the field of image compression technology, and more specifically, to a semantic information-assisted task-oriented image compression method, system and device. Background Art

[0002] Drones usually use aerial cameras and remote sensing systems to collect images of various scenes. During the aerial photography process, drones must forward image data to the control end in real time for analysis and monitoring. However, with the expansion of aerial coverage and the improvement of image quality and accuracy, the amount of image data has increased dramatically.

[0003] In order to solve the problem of limited bandwidth in UAV communication, image compression algorithms suitable for UAV communication have emerged. Through real-time image compression, the amount of data is greatly reduced, reducing the demand for wireless communication bandwidth. Traditional image compression algorithms, such as JPEG and JPEG2000, have reached a performance bottleneck due to the inability to obtain deeper semantic information in the image. In addition, the widespread application of high-resolution image sensors requires a higher compression ratio without affecting the quality of image reconstruction.

[0004] Traditional image compression methods usually aim to achieve pixel-level consistency without paying attention to the performance of subsequent tasks. Taking drone aerial photography as an example, the images sent back are often used for intelligent tasks such as target detection and image classification. At high compression rates, traditional compression schemes may cause distortion of image structure and color, which is likely to lead to a sharp decline in the performance of subsequent tasks. Therefore, traditional image compression algorithms are difficult to meet the processing requirements of drone images, which poses a challenge to drone communications.

[0005] In recent years, with the development of artificial intelligence and computer hardware, semantic segmentation algorithms have been proposed, which can extract semantic information by analyzing various semantic features inside the image, such as objects, background, and relationship information. Based on the semantic information, different regions in the image are processed with different compression rates, which can effectively retain the task-related semantic information in the image, achieve semantic-level image consistency, and effectively ensure the efficiency of subsequent tasks. Therefore, it has the potential to solve the problem of high-quality image transmission under the condition of limited available communication bandwidth in UAV wireless communication systems. Therefore, researchers have provided a series of image semantic communication methods based on intelligent algorithms such as deep learning and convolutional neural networks, which achieve higher compression rates by extracting image semantic information.

[0006] However, this image semantic communication method based on intelligent algorithms still has the following defects: it cannot meet dynamically changing task requirements. For example, the specific requirements of the drone aerial photography task (the target of attention) may change with the progress of the aerial photography. In other words, the task requirements are dynamically changing, which may cause the image semantic communication method to be unable to correctly transmit information related to the new task requirements. Summary of the invention

[0007] In order to solve the problem of limited bandwidth in UAV communication, the present invention provides a semantic information-assisted task-oriented image compression method, system and device, which improve the image compression rate while retaining the key semantics of the original image, and can correctly transmit information related to new task requirements when the task requirements are dynamically changing.

[0008] To achieve the above object, according to a first aspect of the present invention, a semantic information-assisted task-oriented image compression method is provided, the method comprising:

[0009] According to the task knowledge base, the original image is semantically segmented to obtain several task-related information units and background information units. The task knowledge base includes task background, task name, task requirements and task relevance score;

[0010] Calculate the weight of each task-related information unit according to the task-relatedness score and data size of each task-related information unit;

[0011] Allocate bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit;

[0012] According to the bandwidth allocation result, image compression is performed on each task-related information unit and background information unit to obtain a multi-scale compressed image.

[0013] Furthermore, bandwidth is allocated to each task-related information unit and background information unit according to the weight of each task-related information unit, including allocating a preset proportion of the total signal transmission bandwidth to the background information unit; and according to the weight of each task-related information unit, allocating the remaining proportion of the total signal transmission bandwidth other than the preset proportion to each task-related information unit.

[0014] Furthermore, according to the weight of each task-related information unit, the remaining proportion of the total signal transmission bandwidth other than the preset proportion is allocated to each task-related information unit, including calculating the product of the weight of each task-related information unit and the remaining proportion of the total signal transmission bandwidth to obtain the bandwidth of each task-related information unit; when the bandwidth of each task-related information unit is not less than its data size, the bandwidth of the task-related information unit is kept unchanged; when the bandwidth of each task-related information unit is less than its data size, the difference between the data size of the task-related information unit and its bandwidth is allocated to the background information unit.

[0015] Furthermore, the weight of each task-related information unit is calculated based on the task relevance score and data size of each task-related information unit, including calculating the product of the task relevance score and the data size of each task-related information unit; calculating the ratio of the product of each task-related information unit to the sum of several products to obtain the weight of each task-related information unit.

[0016] Furthermore, according to the task knowledge base, the original image is semantically segmented to obtain a number of task-related information units and background information units, including using an image semantic segmentation model to perform semantic segmentation on the original image according to the task requirements provided by the task knowledge base, and deconstructing the original image into a number of task-related information units and background information units.

[0017] Furthermore, the image semantic segmentation model is jointly composed of the Yolo-World algorithm and the Segment Anything Model algorithm. Through the image semantic segmentation model, according to the task requirements provided by the task knowledge base, the original image is semantically segmented, and the original image is deconstructed into a number of task-related information units and background information units, including using the domain knowledge base as a training set for the Yolo-World algorithm to train the Yolo-World algorithm, and the domain knowledge base includes image data belonging to different fields and annotated with category labels; through the trained Yolo-world algorithm, according to the task requirements provided by the task knowledge base, target detection is performed on the original image to obtain detection results, and the detection results include task-related target areas; through the Segment Anything Model algorithm, the target in the target area is segmented into independent information units to obtain a number of task-related information units; the part of the original image except for a number of task-related information units is used as the background information unit.

[0018] Furthermore, according to the bandwidth allocation result, image compression is performed on each task-related information unit and background information unit to obtain a multi-scale compressed image, including using an image compression algorithm to compress the image at different compression rates according to the bandwidth allocation result to obtain a multi-scale compressed image.

[0019] According to a second aspect of the present invention, there is also provided a semantic information-assisted task-oriented image compression system, comprising:

[0020] A semantic segmentation module is used to perform semantic segmentation on the original image according to the task knowledge base to obtain a number of task-related information units and background information units. The task knowledge base includes task background, task name, task requirements and task relevance score;

[0021] A weight calculation module, used to calculate the weight of each task-related information unit according to the task relevance score and data size of each task-related information unit;

[0022] A bandwidth allocation module, used to allocate bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit;

[0023] The image compression module is used to compress the image of each task-related information unit and the background information unit according to the bandwidth allocation result to obtain a multi-scale compressed image.

[0024] According to the third aspect of the present invention, there is also provided a semantic information-assisted task-oriented image compression device, which comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any one of the above methods.

[0025] According to a fourth aspect of the present invention, there is also provided a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0026] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0027] The present invention provides a semantic information-assisted task-oriented image compression method, which embeds task requirements in image compression by establishing a task knowledge base; semantically segmenting the original image according to the task requirements to obtain a plurality of task-related information units and background information units; calculating the weight of each task-related information unit according to the task relevance score and data size of each task-related information unit; allocating bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit, thereby realizing bandwidth allocation based on task relevance; and compressing each task-related information unit and background information unit according to the bandwidth allocation result to obtain a multi-scale compressed image, thereby improving the image compression rate while retaining the key semantics of the original image, achieving the purpose of realizing high-quality image transmission under the condition of limited communication bandwidth, and being able to correctly transmit information related to new task requirements when the task requirements are dynamically changing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A flowchart of a semantic information-assisted task-oriented image compression method provided by an embodiment of the present application;

[0030] Figure 2 A flowchart of a semantic information-assisted task-oriented image compression method provided in another embodiment of the present application;

[0031] Figure 3 A schematic diagram of the execution process of a semantic information-assisted task-oriented image compression method provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of a task knowledge base provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0036] like Figure 1 As shown, a semantic information-assisted task-oriented image compression method is provided, which can be executed by a terminal or by a server communicating with the terminal through a network. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc. The server can be an independent server, or a server cluster composed of multiple servers can be used. Taking the method applied to a terminal as an example, the method includes the following steps:

[0037] Step 101 , semantically segmenting the original image according to the task knowledge base to obtain a number of task-related information units and background information units.

[0038] The task knowledge base includes task background, task name, task requirements and task relevance score. Task background is the scene corresponding to the task, such as drone aerial photography, satellite remote sensing, etc. Task requirements are the requirements corresponding to the task background and task name, such as urban traffic monitoring, behavior assessment, etc. Task relevance score is the score corresponding to the task requirement, which can be obtained through expert scoring, for example.

[0039] Exemplarily, the terminal performs semantic segmentation on the original image through an image semantic segmentation model according to the task requirements provided by the task knowledge base, and deconstructs the original image into a number of task-related information units and background information units.

[0040] In an embodiment in which the image semantic segmentation model is jointly composed of the Yolo-World algorithm and the Segment Anything Model algorithm, a domain knowledge base is used as a training set of the Yolo-World algorithm to train the Yolo-World algorithm, and the domain knowledge base includes image data belonging to different domains and annotated with category labels; through the trained Yolo-world algorithm, according to the task requirements provided by the task knowledge base, target detection is performed on the original image to obtain the detection result, and the detection result includes the task-related target area; through the Segment Anything Model algorithm, the target in the target area is segmented into independent information units to obtain a number of task-related information units; the part of the original image except for the several task-related information units is used as the background information unit.

[0041] Step 102 , calculating the weight of each task-related information unit according to the task-related information score and data size of each task-related information unit.

[0042] Exemplarily, the terminal calculates the product of the task relevance score and the data size of each task related information unit; calculates the ratio of the product of each task related information unit to the sum of several products to obtain the weight of each task related information unit.

[0043] Step 103: Allocate bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit.

[0044] Exemplarily, the terminal allocates a preset proportion of the total signal transmission bandwidth to the background information unit; and allocates the remaining proportion of the total signal transmission bandwidth except the preset proportion to each task-related information unit according to the weight of each task-related information unit.

[0045] Considering that the data size of each task-related information unit is different, when the bandwidth allocated to each task-related information unit is smaller than the data size, the excess bandwidth needs to be fed back to the background information unit. Specifically, according to the weight of each task-related information unit, the remaining proportion of the total bandwidth for signal transmission other than the preset proportion is allocated to each task-related information unit, including calculating the product of the weight of each task-related information unit and the remaining proportion of the total bandwidth for signal transmission other than the preset proportion to obtain the bandwidth of each task-related information unit; when the bandwidth of each task-related information unit is not smaller than its data size, the bandwidth of the task-related information unit is kept unchanged; when the bandwidth of each task-related information unit is smaller than its data size, the difference between the data size of the task-related information unit and its bandwidth is allocated to the background information unit.

[0046] Step 104 , according to the bandwidth allocation result, image compression is performed on each task-related information unit and background information unit to obtain a multi-scale compressed image.

[0047] Among them, the multi-scale compressed image is a compressed image with different compression rates in different regions based on semantic information.

[0048] Exemplarily, the terminal uses an image compression algorithm to compress the image of each task-related information unit and the background information unit at different compression rates according to the bandwidth allocation result, so as to obtain a multi-scale compressed image.

[0049] In the above-mentioned semantic information-assisted task-oriented image compression method, task requirements are embedded in image compression by establishing a task knowledge base; according to the task requirements, the original image is semantically segmented to obtain a number of task-related information units and background information units; the weight of each task-related information unit is calculated according to the task relevance score and data size of each task-related information unit; according to the weight of each task-related information unit, bandwidth is allocated to each task-related information unit and background information unit, thereby realizing bandwidth allocation based on task relevance; according to the bandwidth allocation result, image compression is performed on each task-related information unit and background information unit to obtain a multi-scale compressed image, thereby retaining the key semantics of the original image while improving the image compression rate, achieving the purpose of realizing high-quality image transmission under the condition of limited communication bandwidth, and when the task requirements are dynamically changing, the information related to the new task requirements can also be correctly transmitted.

[0050] In one embodiment, Figure 2 As shown in the figure, for the UAV image transmission scenario with specific tasks, a semantic information-assisted task-oriented image compression method is provided, and a task knowledge base is established to realize the update mechanism to adapt to the dynamic changes of the task.

[0051] The task knowledge base consists of the task background, task name, task requirements and task relevance scores of drone aerial photography. The task requirements mainly provide guidance for image semantic segmentation and drive image semantic segmentation to deconstruct the image into several task-related information units; the task relevance score provides information support for adaptive bandwidth allocation, and its main goal is to achieve bandwidth allocation under the guidance of task relevance.

[0052] The domain knowledge base is the knowledge embedded in the model parameters during the training process of the image semantic segmentation algorithm. It mainly includes annotated image data and is used to implement the training and optimization of the image semantic segmentation algorithm.

[0053] The image semantic segmentation module is composed of the Yolo-World algorithm and the Segment Anything Model (SAM) algorithm. Yolo-World is used to realize target detection under task requirements (prompt words in text form) and obtain the task-related target area contained in the image; then SAM is used to segment the targets in the target area into independent information units, and finally the image is deconstructed into background information units and several task-related information units.

[0054] The adaptive bandwidth allocation module combines the task relevance score and data size of the task related information unit to calculate the weight of each task related information unit, as shown in formula (1).

[0055]

[0056] In formula (1), δ i and Size i are the task relevance score and data size of the i-th task-related information unit, and n is the number of deconstructed task-related information units.

[0057] The multi-scale image compression module is used to compress the deconstructed task-related information units and background information units with different compression rates in turn according to the allocated bandwidth, and finally form a multi-scale compressed image with different compression rates in different areas to retain the task-related areas to the maximum extent.

[0058] like Figure 3 As shown in FIG. 1 , the execution process of a semantic information-assisted task-oriented image compression method is as follows:

[0059] Step 1, algorithm initialization, such as Figure 4 As shown in the figure, based on the UAV aerial photography mission scenario (taking urban traffic monitoring as an example), the mission information and domain information are analyzed, and the mission knowledge base and domain knowledge base are established (as training sets for adjusting the Yolo-World model). Figure 4 For example, the task requirements of urban traffic monitoring are set to people and cars, and the task relevance scores are set to 1 and 2 respectively. After completing the deployment of the task knowledge base and fine-tuning of the algorithm model, proceed to step 2.

[0060] Step 2: Input the original image and task requirements into the Yolo-World algorithm for target detection to obtain the area containing people and cars in the original image. Then input the detection results into SAM for semantic segmentation to deconstruct the original image into several task-related information units. After completing the image semantic segmentation, proceed to step 3.

[0061] Step 3, reserve 10% of the total signal transmission bandwidth for the background information unit; input the task relevance score provided by the task knowledge base, the data size of the task related information unit and the available bandwidth (i.e., 90% of the total signal transmission bandwidth) into the adaptive bandwidth allocation module, and allocate bandwidth to each information unit according to formula (1); in the allocation process, after allocating bandwidth to each task related information unit, the excess bandwidth (i.e., the difference between the data size of the task related information unit and its bandwidth when the bandwidth of the task related information unit is less than its data size) is fed back to the background information unit. After completing the adaptive bandwidth allocation, execute step 4.

[0062] Step 4: The bandwidth allocation results and information units are input into the multi-scale image compression module. Various image compression algorithms such as JPEG, JPEG2000, and PNG can be used (the demonstration process uses the JPEG image compression scheme). According to the bandwidth allocation results, different scales of images are compressed in different areas to form a multi-scale compressed image with different compression rates in different areas, so as to retain the task-related areas to the maximum extent.

[0063] A semantic information-assisted task-oriented image compression method provided in this embodiment compresses images of different scales by region based on semantic information, which can retain task-related information to the greatest extent possible; by establishing a task knowledge base and designing a corresponding interaction mechanism, it is ensured that the image compression algorithm can effectively compress images under the dynamically changing task background during drone aerial photography.

[0064] After testing, a semantic information-assisted task-oriented image compression method provided in this embodiment is used on the semantic drone dataset released by Graz University of Technology. The original image and the compressed image are simultaneously input into Yolo-World for target detection (confidence is set to 60%). The image compression rate can reach nearly 3%, while ensuring a target detection rate of more than 90%, achieving effective image compression with a high compression rate while retaining mission-related area information.

[0065] The present application also provides a semantic information-assisted task-oriented image compression system, which includes a semantic segmentation module, a weight calculation module, a bandwidth allocation module and an image compression module.

[0066] Among them, the semantic segmentation module is used to perform semantic segmentation on the original image according to the task knowledge base to obtain several task-related information units and background information units. The task knowledge base includes task background, task name, task requirements and task relevance score.

[0067] The weight calculation module is used to calculate the weight of each task-related information unit according to the task relevance score and data size of each task-related information unit.

[0068] The bandwidth allocation module is used to allocate bandwidth to each task related information unit and background information unit according to the weight of each task related information unit.

[0069] The image compression module is used to compress the image of each task-related information unit and the background information unit according to the bandwidth allocation result to obtain a multi-scale compressed image.

[0070] The present application also provides a semantic information-assisted task-oriented image compression device, which may be a computer device, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a task-oriented image compression method assisted by semantic information is implemented.

[0071] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0072] like Figure 5 As shown, the present application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps in the above-mentioned method embodiments.

[0073] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0074] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0075] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

[0077] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A semantic information-assisted task-oriented image compression method, characterized in that: include: According to the task knowledge base, semantic segmentation is performed on the original image to obtain a number of task-related information units and background information units, wherein the task knowledge base includes task background, task name, task requirements and task relevance score; Calculate the weight of each task-related information unit according to the task-relatedness score and data size of each task-related information unit; Allocating bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit; According to the bandwidth allocation result, image compression is performed on each task-related information unit and background information unit to obtain a multi-scale compressed image.

2. The method according to claim 1, characterized in that The allocating bandwidth to each task-related information unit and the background information unit according to the weight of each task-related information unit includes: Allocating a preset proportion of the total signal transmission bandwidth to the background information unit; According to the weight of each task related information unit, the remaining proportion of the total signal transmission bandwidth except the preset proportion is allocated to each task related information unit.

3. The method according to claim 2, characterized in that The allocating the remaining proportion of the total signal transmission bandwidth except the preset proportion to each task-related information unit according to the weight of each task-related information unit includes: Calculate the product of the weight of each task-related information unit and the remaining proportion of the total signal transmission bandwidth to obtain the bandwidth of each task-related information unit; When the bandwidth of each task-related information unit is not less than its data size, the bandwidth of the task-related information unit is kept unchanged; In the case that the bandwidth of each task-related information unit is smaller than its data size, the difference between the data size of the task-related information unit and its bandwidth is allocated to the background information unit.

4. The method according to claim 1, characterized in that Calculating the weight of each task-related information unit according to the task-related information unit's task-related information score and data size includes: Calculate the product of the task relevance score and the data size of each task-related information unit; The ratio of the product of each task-related information unit to the sum of several products is calculated to obtain the weight of each task-related information unit.

5. The method according to claim 1, characterized in that According to the task knowledge base, the original image is semantically segmented to obtain several task-related information units and background information units, including: Through the image semantic segmentation model, according to the task requirements provided by the task knowledge base, the original image is semantically segmented and deconstructed into several task-related information units and background information units.

6. The method according to claim 5, characterized in that The image semantic segmentation model is composed of the Yolo-World algorithm and the Segment Anything Model algorithm. The image semantic segmentation model performs semantic segmentation on the original image according to the task requirements provided by the task knowledge base, and deconstructs the original image into several task-related information units and background information units, including: The Yolo-World algorithm is trained by using a domain knowledge base as a training set of the Yolo-World algorithm, wherein the domain knowledge base includes image data belonging to different domains and annotated with category labels; Through the trained Yolo-world algorithm, according to the task requirements provided by the task knowledge base, target detection is performed on the original image to obtain the detection result, which includes the target area related to the task; By using the Segment Anything Model algorithm, the target in the target area is segmented into independent information units to obtain a number of task-related information units; The portion of the original image other than the plurality of task-related information units is used as a background information unit.

7. The method according to claim 1, characterized in that The method of performing image compression on each task-related information unit and background information unit according to the bandwidth allocation result to obtain a multi-scale compressed image includes: Through the image compression algorithm, according to the bandwidth allocation result, different compression rates are used to compress the information units related to each task and the background information units to obtain a multi-scale compressed image.

8. A semantic information-assisted task-oriented image compression system, characterized in that: include: A semantic segmentation module, used to perform semantic segmentation on the original image according to a task knowledge base to obtain a number of task-related information units and background information units, wherein the task knowledge base includes task background, task name, task requirements and task relevance score; A weight calculation module, used to calculate the weight of each task-related information unit according to the task relevance score and data size of each task-related information unit; A bandwidth allocation module, configured to allocate bandwidth to each task-related information unit and background information unit according to the weight of each task-related information unit; The image compression module is used to compress the image of each task related information unit and the background information unit according to the bandwidth allocation result to obtain a multi-scale compressed image.

9. A semantic information-assisted task-oriented image compression device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images