Incremental task-oriented object detection semantic communication system and method

Abstract

The present invention belongs to the field of semantic communication technology and discloses a semantic communication system and method for target detection for incremental tasks; the system comprises: a task preprocessing module determines the number of new categories according to the categories to be detected, and obtains a status signal according to the number of new categories; a semantic encoding module extracts features of a to-be-detected image data set to obtain original features to be identified and current features to be identified when the status signal is an incremental update status signal; a semantic decoding module receives the incremental update status signal, and identifies the original features to be identified and the current features to be identified to obtain new category recognition results and original category recognition results; a parameter optimization module constructs a total loss function according to the recognition results and the new category labels, and optimizes the parameters of the model to complete the target detection of the incremental task; the present invention constructs a total loss function through the recognition results of the original category and the new category, and the total loss function constrains the model, so that the model can accurately identify the original category while accurately identifying the new category.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication technology, and in particular to an incremental task-oriented target detection semantic communication system and method. Background Art

[0002] Existing task-oriented image semantic communication systems only involve relatively simple image classification and image detection. In addition, existing image semantic communication systems do not consider the situation of task or category updates, which makes them only applicable to extremely limited tasks with well-defined and narrow scope.

[0003] However, new tasks inevitably arise during the use of image semantic communication. Therefore, we still need to explore how to use new data to perform task-oriented incremental updates to the semantic encoding and decoding modules for object detection training when new categories appear in the task, so that we can achieve high-accuracy object detection for both old and new categories.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present invention is to provide a target detection semantic communication system for incremental tasks, aiming to solve the technical problem in the prior art that the target detection semantic communication system can accurately detect new categories while the detection accuracy of the original categories cannot be guaranteed.

[0006] To achieve the above objectives, the present invention provides an incremental task-oriented target detection semantic communication system, which includes a task preprocessing module, a semantic encoding module, a wireless channel, a parameter optimization module, and a semantic decoding module;

[0007] The task preprocessing module is used to obtain a dataset of images to be detected and categories to be detected, determine the number of new categories based on the categories to be detected, obtain a status signal based on the number of new categories, and send the status signal and the dataset of images to be detected to the semantic encoding module;

[0008] The semantic encoding module is configured to receive the status signal and, when the status signal is an incremental update status signal, perform feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and transmit the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel;

[0009] The semantic decoding module is configured to receive the original feature to be identified, the current feature to be identified, and the incremental update status signal, and perform feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result;

[0010] The parameter optimization module is used to construct a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

[0011] Optionally, the task preprocessing module is further configured to obtain a task category list, determine whether the task category list is a blank list, and obtain a first training state signal if the task category list is a blank list;

[0012] The task preprocessing module is further configured to, when the task category list is not a blank list, traverse the task category list and match the category to be detected with the task category in the task category list;

[0013] The task preprocessing module is further configured to obtain a test status signal when the category to be detected successfully matches all the task categories in the task category list;

[0014] The task preprocessing module is further configured to obtain a new number of categories based on the number of matching failures between the category to be detected and the task categories in the task category list when there is a matching failure between the category to be detected and the task categories in the task category list, and obtain an incremental update status signal based on the new number of categories.

[0015] Optionally, the semantic encoding module is further configured to, upon receiving an incremental update status signal, copy the feature extraction network model based on the incremental update status signal to obtain an original feature extraction network model and a current feature extraction network model;

[0016] The semantic encoding module is further used to input the image data set to be detected into the original feature extraction network model and the current feature extraction network model at the same time to obtain the original features to be identified and the current features to be identified;

[0017] The semantic encoding module is further configured to send the original feature to be identified, the current feature to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel.

[0018] Optionally, the new category recognition result includes the new category recognition result of the current model, and the original category recognition result includes the original category recognition result of the model and the original category recognition result of the current model;

[0019] The semantic decoding module is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model and the current feature recognition network model;

[0020] The semantic decoding module is further used to input the original features to be identified into the original feature recognition network model to obtain the original model original category recognition result;

[0021] The semantic decoding module is further used to input the current features to be identified into the current feature recognition network model to obtain the new category recognition result of the current model and the original category recognition result of the current model.

[0022] Optionally, the semantic decoding module is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model and the initial current feature recognition network model;

[0023] The semantic decoding module is further configured to add a corresponding number of new categories to the initial existing feature recognition network model according to the number of new categories in the incremental update state signal, wherein the recognition result of the initial existing feature recognition network model includes the original categories and the new categories;

[0024] The semantic decoding module is further used to initialize the parameters corresponding to the newly added categories in the initial current feature recognition network model to obtain the current feature recognition network model.

[0025] Optionally, the parameter optimization module is further configured to construct an original category loss function based on the original model original category recognition result and the current model original category recognition result;

[0026] The parameter optimization module is further used to construct a new category loss function based on the new category recognition result of the current model and the new category label;

[0027] The parameter optimization module is further configured to obtain a total loss function based on the original category loss function, the preset balance parameter, and the new category loss function;

[0028] The parameter optimization module is also used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

[0029] Optionally, the semantic encoding module is further configured to, when the state signal is a first training state signal, initialize parameters of the feature extraction network model based on the first training state signal to obtain an initial feature extraction network model 23;

[0030] The semantic encoding module is further configured to input the image dataset to be detected into the initial feature extraction network model 23 to obtain initial features to be identified;

[0031] The semantic encoding module is further configured to send the initial features to be identified and the status signal to the semantic decoding module via a wireless channel.

[0032] Optionally, the semantic decoding module is further configured to, upon receiving the initial feature to be identified and the first training state signal, initialize the feature recognition network model based on the first training state signal to obtain an initial feature recognition network model 43;

[0033] The semantic decoding module is further configured to input the initial features to be identified into the initial feature recognition model to obtain an initial target detection result.

[0034] Optionally, the parameter optimization module is further configured to construct an initial loss function based on the initial target detection result and the category label of the initial feature to be identified;

[0035] The parameter optimization module is further used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the initial loss function to complete the target detection of the first task.

[0036] Optionally, the semantic encoding module is further configured to, upon receiving a test status signal, input the image dataset to be detected into a feature extraction network model based on the test status signal to obtain a test feature to be identified;

[0037] The semantic encoding module is further configured to send the test feature to be identified and the test status signal to the semantic decoding module via a wireless channel.

[0038] Optionally, the semantic decoding module is further configured to, upon receiving the test feature to be identified and the test status signal, input the training feature to be identified into a feature extraction network model based on the test status signal to obtain a test feature identification result.

[0039] Optionally, the parameter optimization module is further configured to construct a test loss function based on the test feature recognition result and the category label of the training feature to be recognized;

[0040] The parameter optimization module is further used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the test loss function to complete the target detection of the test task.

[0041] Optionally, the feature to be identified includes at least one of the original feature to be identified, the current feature to be identified, the initial feature to be identified, and the test feature to be identified;

[0042] The semantic encoding module is further configured to compress the feature to be identified to obtain a compressed feature to be identified, and send the compressed feature to be identified to the semantic decoding module via a wireless channel;

[0043] The semantic decoding module is further configured to receive the compressed features to be identified, and perform feature decoding on the compressed features to be identified to obtain the features to be identified.

[0044] Optionally, the incremental task-oriented target detection semantic communication method is applied to the incremental task-oriented target detection semantic communication system, and the method includes:

[0045] The task preprocessing module obtains a dataset of images to be detected and categories to be detected, determines the number of new categories according to the categories to be detected, obtains a status signal according to the number of new categories, and sends the status signal and the dataset of images to be detected to the semantic encoding module;

[0046] The semantic encoding module receives the status signal, and when the status signal is an incremental update status signal, performs feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and sends the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel;

[0047] The semantic decoding module receives the original feature to be identified, the current feature to be identified, and the incremental update status signal, and performs feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result;

[0048] The parameter optimization module constructs a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimizes the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the incremental update target detection training.

[0049] In addition, to achieve the above-mentioned purpose, the present invention further proposes a target detection semantic communication method for incremental tasks, which is applied to the target detection semantic communication system. The method includes:

[0050] The task preprocessing module obtains a dataset of images to be detected and categories to be detected, determines the number of new categories according to the categories to be detected, obtains a status signal according to the number of new categories, and sends the status signal and the dataset of images to be detected to the semantic encoding module;

[0051] The semantic encoding module receives the status signal, and when the status signal is an incremental update status signal, performs feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and sends the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel;

[0052] The semantic decoding module receives the original feature to be identified, the current feature to be identified, and the incremental update status signal, and performs feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result;

[0053] The parameter optimization module constructs a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimizes the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the incremental update target detection training.

[0054] When there is a new category in the category to be detected input by the task processing module, the present invention obtains an incremental update status signal when there is a new category. After state control of the semantic encoding module and the semantic decoding module in the system, feature extraction and feature recognition are performed on the input image set to obtain new category recognition results and original category recognition results. Further, based on the new category recognition results and the original category recognition results, a total loss function that simultaneously constrains the new category recognition accuracy and the original category recognition accuracy is constructed. According to the total loss function, the models in the semantic encoding module and the semantic decoding module are optimized. The optimized model can accurately identify incremental tasks that include both new categories and original categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a schematic diagram of the system structure of the first embodiment of the incremental task-oriented target detection semantic communication system of the present invention;

[0056] Figure 2 This is a schematic diagram of the overall data transmission process of an embodiment of the incremental task-oriented target detection semantic communication system of the present invention;

[0057] Figure 3 This is a schematic diagram of a task preprocessing module of an embodiment of an incremental task-oriented object detection semantic communication system of the present invention;

[0058] Figure 4 This is a schematic diagram of the overall model of the target detection network model of an embodiment of the target detection semantic communication system for incremental tasks of the present invention;

[0059] Figure 5 This is a schematic diagram of the system structure of the second embodiment of the incremental task-oriented target detection semantic communication system of the present invention;

[0060] Figure 6 This is a schematic diagram of a feature extraction model for a semantic encoding module of an incremental task-oriented semantic communication system according to an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of a feature recognition model of a semantic decoding module according to an embodiment of the object detection method of the present invention;

[0062] Figure 8 This is a schematic diagram of the system structure of the third embodiment of the incremental task-oriented object detection semantic communication system of the present invention;

[0063] Figure 9 This is a schematic diagram of the system structure of the fourth embodiment of the incremental task-oriented object detection semantic communication system of the present invention;

[0064] Figure 10 This is a schematic diagram of feature compression of a semantic encoding module of an embodiment of an incremental task-oriented semantic communication system of the present invention;

[0065] Figure 11 This is a schematic diagram of feature recovery of a semantic decoding module of an embodiment of an incremental task-oriented semantic communication system of the present invention;

[0066] Figure 12 This is a flowchart of the first embodiment of the target detection semantic communication method for incremental tasks of the present invention.

[0067] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0068] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0069] The embodiment of the present invention provides a target detection semantic communication system for incremental tasks, referring to Figure 1 , Figure 1 This is a system structure diagram of the first embodiment of an incremental task-oriented target detection semantic communication system of the present invention.

[0070] The target detection semantic communication system for incremental tasks includes a task preprocessing module 10, a semantic encoding module 20, a wireless channel 30, a semantic decoding module 40 and a parameter optimization module 50;

[0071] It can be understood that the semantic decoding module 40 and the semantic encoding module 20 correspond to each other, the feature extraction model is set in the semantic encoding module 20, and the feature recognition model is set in the semantic decoding module 40; wherein, the semantic decoding module 40 and the semantic encoding module 20 transmit data stream signals and status stream signals through the wireless channel 30.

[0072] In the specific implementation, the transmission of state flow and data flow can refer to Figure 2 In the figure, after receiving a task, the system preprocesses the task to determine the state of the semantic encoding module 20. After determining the state, data stream transmission is performed. A task is defined as the current batch of image datasets and the target category to be detected. For example, Task 1 is to detect pedestrians and cars in the current batch of images. When a task arrives, the system first determines the state of the semantic encoding module 20 and semantic decoding module 40 through the state stream, and then transmits the data stream.

[0073] It is worth noting that the traditional communication system: a communication system in which the source and channel are encoded separately. This system has a "cliff effect" when transmitting images, that is, when the channel conditions are poor, the performance will suddenly drop to an unusable state. The semantic decoding module 40 cannot restore meaningful images, and thus cannot perform downstream target detection tasks. Therefore, based on the target detection task, the present application deploys the front end of the deep learning model for target detection (which can be understood as a feature extraction neural network) on the edge sending side, and the back end (which can be understood as a feature recognition network model) on the cloud receiving side. The task-oriented semantic communication method can achieve a higher compression rate, thereby saving bandwidth. The semantic encoding module 20 extracts the semantic features of the image and compresses them before transmitting them in the wireless channel 30. The semantic decoding module 40 restores the compressed semantic features and inputs them into the model back end to output the target detection results.

[0074] It should be emphasized that existing task-oriented image semantic communication systems only involve relatively simple image classification and image detection, and do not consider the situation where tasks or categories are updated. As a result, they are only applicable to a very limited number of tasks with clear and narrow definitions. In this application, when new categories appear in the task, the model backend of the semantic decoding module 40 is updated and incremental update training is performed using the total loss function, so that the new model can perform high-precision detection on both old and new categories.

[0075] The task preprocessing module 10 is used to obtain the image dataset to be detected and the category to be detected, determine the number of new categories based on the category to be detected, obtain a status signal based on the number of new categories, and send the status signal and the image dataset to be detected to the semantic encoding module 20.

[0076] It is understandable that the image dataset to be detected can be image data manually input into the incremental task-oriented target detection semantic communication system, and the category to be detected can be the category selected by the user to be identified in the target detection of the image dataset to be detected.

[0077] In a specific implementation, the image dataset to be detected can be 1,000 images input by the user, and the categories to be detected can be the target detection semantic communication system for incremental tasks that wants to identify cars, furniture, puppies, and trees in these 100 images. In this case, cars, furniture, puppies, and trees are the categories to be detected.

[0078] It should be understood that the status signal can be a system status obtained based on whether there is a new category in the category to be detected (the system status here can also be further understood as the status of the semantic encoding module 20 and the status of the semantic decoding module 40. After obtaining the signal corresponding to the system status, the signal is sent to the semantic encoding module 20 and the semantic decoding module 40. The semantic encoding module 20 and the semantic decoding module 40 can adjust their own status according to the status signal).

[0079] It's understandable that new categories can be understood as categories not already in the task category list. The task category list can be understood as a list of task categories to be tested each time. For example, the task category list includes cars, furniture, houses, and people; the categories to be tested this time include cars, furniture, puppies, and trees. Therefore, the categories of puppies and trees are new categories, and the number of new categories is 2.

[0080] It should be noted that the status signal obtained according to the number of new categories can be used to determine whether the task category list is a blank list, that is, whether there are categories that have been identified in the list. If the task category list is a blank list, the status signal obtained is the initial state, and the initial state here can be understood as the first time of target detection; if the task category list is not a blank list, traverse the task category list and match the category to be detected with the task category in the task category list; if all matches are successful, the status signal obtained is the training state, and all matches are successful, that is, the categories to be detected this time are all categories that have been detected, and there are no new categories that need to be detected; if there is a matching failure, the existence of matching recognition can be understood as one or more categories to be detected are not found in the task category list, then the status signal obtained is the incremental update state, and the number of new categories is obtained according to the number of failed matches between the category to be detected and the task category in the task category list.

[0081] It should be noted that the status signal includes the incremental update status signal, the first test status signal, and the test status signal. The status signal is determined according to the number of new categories. It can be understood that the number of new categories is included in the status signal, and the number of new categories of the first test status signal and the test status signal is 0.

[0082] In the specific implementation, please refer to Figure 3 The preprocessing step in the figure can also be understood as the step of determining the status signal. Task preprocessing module 10 searches the target category list and monitors port 1 for incoming target detection tasks. Port 2 receives the image data to be detected. When a detection task arrives, the image dataset to be detected for that task is obtained.

[0083] If the target category list is empty, it indicates the first time a task has been received. The state is set to "0 - First Training State" and output from port 3, along with the number of new categories, 0. The dataset is divided into training and test sets, which are output from port 4. The target category set to be detected is stored in the target category list.

[0084] If the target category list is not empty, the task's target category is compared with the stored category list. If the target category is already in the target category list, the system state is determined to be "1-test state" and output from port 3 along with the new category number 0. The data set is output from port 4 as a test set for transmission.

[0085] If the category list is not empty, compare the target categories to be detected for this task with the task category list. If any of the target categories are not in the task category list, record them as new categories and add them to the task category list. Set the system state to "2-Incremental Update State" and output this state and the number of new categories, n, from port 3. Split the dataset into training and test sets, which are output from port 4. Add the new category set to the target category list.

[0086] The semantic encoding module 20 is used to receive the status signal. When the status signal is an incremental update status signal, the semantic encoding module 20 performs feature extraction on the image data set to be detected based on the incremental update status signal to obtain the original features to be identified and the current features to be identified, and sends the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module 40 through the wireless channel 30.

[0087] In a specific implementation, assuming that the input is an image x and the output is a semantic feature matrix y, the neural network can be expressed by the following formula:

[0088] y=T θ (x)

[0089] Among them, T θ (·) represents the mapping function of the feature extraction network model of the semantic encoding module 20.

[0090] It should be noted that the reference Figure 4 , Semantic Encoding Module 20T for Incremental Task-Oriented Object Detection Semantic Communication System θ , 3 channels, 4 semantic decoding modules 40 neural network R Ψ Semantic encoding module 20 neural network T θ , channel and semantic decoding module 40 neural network R Ψ Deep neural network SC θ,Ψ Joint training: A non-trainable noise layer between the semantic encoding module 20 and the semantic decoding module 40.

[0091] The semantic feature matrix y obtained after feature extraction by the semantic coding module 20 is degraded into the matrix after being affected by channel noise and interference. Please refer to the following formula for details:

[0092]

[0093] Where h represents the channel parameter and η represents Gaussian noise. This module simulates the effects of a real wireless channel 30 on the signal, which may include Gaussian noise, slow fading, and multipath effects.

[0094] Among them, the semantic decoding module 40 receives the image features After decompression, the target detection result z = {z1, z2, z3} is obtained through the feature recognition network model. The feature recognition network model can be expressed by the following formula:

[0095]

[0096] Among them, R ψ (·) represents the mapping function of the semantic decoding module 40. Z1 represents the heat map (category) prediction result, Z2 represents the center point prediction result, and Z3 represents the width and height prediction result.

[0097] It is understandable that the semantic encoding module 20 includes a feature extraction network model, and the feature extraction network extracts features from the image dataset to be detected according to the category to be detected, and obtains original features to be identified and current features to be identified.

[0098] It is understandable that the original features to be identified may be features obtained by extracting features from categories existing in the task category list, and the current features to be identified may be features obtained by extracting features from new categories not existing in the task category list.

[0099] It should be understood that the feature extraction model can be a resnet50 network model in the classic deep learning model.

[0100] The semantic decoding module 40 is used to receive the original feature to be identified, the current feature to be identified, and the incremental update status signal, and perform feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and a test feature recognition result.

[0101] It is understandable that the semantic decoding module 40 includes a feature recognition network model, and the feature recognition network performs feature recognition on the original features to be recognized and the current features to be recognized according to the category to be detected, and obtains a new category recognition result and a test feature recognition result.

[0102] It should be understood that the new category recognition result is the result obtained by identifying the new category from the existing features to be identified based on the feature recognition network model; the test feature recognition result is the result obtained by identifying the original category from the existing features to be identified based on the feature recognition network model.

[0103] It should be noted that the feature recognition model can be a network model composed of the feature recognition part of the classic deep learning model.

[0104] The parameter optimization module 50 is used to construct a total loss function based on the new category recognition results, the test feature recognition results and the new category labels of the image data set to be detected, and optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

[0105] It should be noted that the loss function for the new category is constructed based on the new category recognition results and the new category labels of the image dataset to be tested. The loss function for the new category can constrain the new category detection results in the feature extraction and feature recognition models, thereby improving the recognition accuracy of the new category. The loss function for the original category is constructed based on the test feature recognition results. The loss function for the original category can constrain the original category detection results in the feature extraction and feature recognition models, thereby improving the recognition accuracy of the original category.

[0106] It should be further explained that by constructing a total loss function based on the loss function of the new category and the loss function of the original category, it is possible to simultaneously constrain the original category detection results and the new category detection results in the feature extraction and feature recognition model, thereby improving the recognition accuracy of the new category and the original category.

[0107] This embodiment obtains the output state according to the list of categories to be detected and task categories, and sends the status signal to the encoder and decoder according to different status signals, so that the encoder and decoder can perform feature extraction and feature recognition according to different status signals, and further train the network model in the encoder and decoder according to the recognition results, so as to more accurately identify various types of data to be detected.

[0108] The embodiment of the present invention provides a target detection semantic communication system for incremental tasks, referring to Figure 5 , Figure 5 This is a system structure diagram of the first embodiment of an incremental task-oriented target detection semantic communication system of the present invention.

[0109] The semantic encoding module 20 in the incremental task-oriented target detection semantic communication system also includes an original feature extraction network model 21 and a prior feature extraction network model; the semantic decoding module 40 also includes an original feature recognition network model 41 and a current feature recognition network model 42.

[0110] The semantic encoding module 20 is further configured to, upon receiving an incremental update status signal, copy the feature extraction network model based on the incremental update status signal to obtain an original feature extraction network model 21 and a current feature extraction network model 22;

[0111] It is understandable that when the state signal is in the incremental update state, the feature extraction network model is copied to obtain the current feature extraction network model 22 and the original feature extraction network model 21.

[0112] It should be understood that the current feature extraction network model 22 can be used to extract features of the new category and features of the original category; the original feature extraction network model 21 can be used to extract features of the original category.

[0113] The semantic encoding module 20 is further configured to simultaneously input the image data set to be detected into the original feature extraction network model 21 and the current feature extraction network model 22 to obtain the original features to be identified and the current features to be identified;

[0114] It is understandable that the current features to be identified may include features of the new category as well as features of the original category, and the original features to be identified may be features of the original category.

[0115] It should be noted that when the status signal is in the incremental update state, the current feature extraction network model 22 of the semantic encoding module 20 and the current feature recognition network model 42 of the semantic decoding module 40 perform corresponding transmission of the current features to be identified; the original feature extraction network model 21 of the semantic encoding module 20 and the original feature recognition network model 41 of the semantic decoding module 40 perform corresponding transmission of the original features to be identified.

[0116] The semantic encoding module 20 is further configured to send the original feature to be identified, the current feature to be identified, and the incremental update status signal to the semantic decoding module 40 via the wireless channel 30 .

[0117] In the specific implementation, please refer to Figure 6 In the figure, the semantic encoding module 20 monitors the system status at port 1. The status signal is defined as {A, B}, where A represents the system status and takes the value of {0, 1, 2}; B represents the number of new categories and takes the value of 0 or n (n is a positive integer).

[0118] When {0, 0} is received, the state of semantic encoding module 20 is set to 0 and sent out of port 3. The model parameters are initialized and set to the training state before the first training. Branch 2.2 is activated, meaning that data is output from port 5 only after passing through the current model.

[0119] When {1, 0} is received, the state of semantic encoding module 20 is set to 1 and sent out of port 3. The current model is set to test state and the parameters are not updated. Branch 2.2 is activated, that is, the data is output from port 5 only after passing through the current model.

[0120] When {2, n} is received, the state of semantic encoding module 20 is set to 2 and sent out port 3. Before training begins, the current model is copied and stored as the old model, and the old model is set to the test state. The current model is set to the training state, and incremental training is performed to update the parameters. Branches 2.1 and 2.2 are both activated, meaning that data is output from port 4 through the old model in the test state and from port 5 through the current model in training.

[0121] The semantic decoding module 40 is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model 41 and the current feature recognition network model 42;

[0122] It should be understood that the incremental update status signal may be a type of status signal of the semantic encoding module 20 , and the status signal may also include a first test status signal and a test status signal, etc.

[0123] It is understandable that when receiving different status signals, the semantic encoding module 20 will adjust its own status accordingly, and different statuses correspond to different feature recognition network models.

[0124] It should be noted that when the incremental update status signal is obtained, when the current feature recognition network model 42 and the original feature recognition network model 41 are copied, the original feature recognition network model 41 is adjusted to the test state, and the current feature recognition network model 42 is adjusted to the training state.

[0125] It should be further explained that in the test state, the parameters in the neural network may not be adjusted, and in the training state, the parameters in the neural network may need to be partially or fully adjusted. The adjustment may be initialization or setting fixed parameters.

[0126] It should be understood that the incremental update status signal may mean that the task category that needs to be identified at this time includes a new category.

[0127] It should be noted that when the semantic decoding module 40 receives the incremental update state signal, the feature recognition network in the semantic decoding module 40 is copied to obtain two identical feature recognition networks, one of which is the current feature recognition network model 42 and the other is the original feature recognition network model 41. Figure 7In the figure, sequence number 1 represents the received state flow signal, sequence number 2 represents the received feature to be identified, and the features to be identified in this embodiment include the current features to be identified and the original features to be identified. Among them, link 2.1 corresponds to the old model, that is, the original feature identification network model 41, and link 2.2 corresponds to the current model, that is, the current feature identification network model 42. Sequence number 3 represents the recognition result of the original feature identification network model 41 for the original feature to be identified, and sequence number 4 represents the recognition result of the current feature identification network model 42 for the new feature to be identified.

[0128] The semantic decoding module 40 is also used to input the original features to be identified into the original feature recognition network model 41 to obtain the original model test feature recognition result; the semantic decoding module 40 is also used to input the current features to be identified into the current feature recognition network model 42 to obtain the current model new category recognition result and the current model test feature recognition result.

[0129] It should be noted that the new category recognition result includes the new category recognition result of the current model, and the test feature recognition result includes the test feature recognition result of the original model and the test feature recognition result of the current model.

[0130] It is understandable that the current feature to be identified may be a feature that has been successfully extracted by the previous feature recognition network model sent by the semantic encoding module 20.

[0131] It should be understood that after the current feature to be identified is input into the current feature identification network model 42, an identification result containing a new category and the original category can be obtained, while inputting the original feature to be identified into the original feature identification network model 41 only obtains the original model test feature recognition result.

[0132] The semantic decoding module 40 is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model 41 and the initial current feature recognition network model 42;

[0133] The semantic decoding module 40 is further configured to add a corresponding number of new categories to the initial feature recognition network model 42 according to the number of new categories in the incremental update state signal, wherein the recognition result of the initial feature recognition network model 42 includes the original categories and the new categories;

[0134] The semantic decoding module 40 is further configured to initialize the parameters corresponding to the newly added categories in the initial current feature recognition network model 42 to obtain the current feature recognition network model 42 .

[0135] In the specific implementation, you can refer to Figure 4In the figure, the semantic decoding module 40 is the semantic decoding module 40. The upper part of the light blue neural network in the semantic decoding module 40 is the original feature recognition network model 41, and the lower part is the current feature recognition network model 42. The circles in the last column of the two neural networks are the recognition categories. The figure shows that there are two new categories, so the output results of two categories are added to the output layer of the current feature recognition network model 42. The corresponding parameters of the new category output results in the neural network are initialized. In simple terms, the number of channels in the category prediction output layer of the current feature recognition network model 42 is updated according to the number of new categories, and the parameters of the added network structure are initialized.

[0136] In a specific implementation, the semantic encoding module 20 neural network T θ , channel and semantic decoding module 40 neural network R Ψ Deep neural network SC θ,Ψ Perform joint training. There is a non-trainable noise layer between the semantic encoding module 20 and the semantic decoding module 40. The input is image x, and the output is semantic feature matrix y. When receiving an object detection task containing a new category: Step 1: Task preprocessing confirms that the status is incremental update. Step 2: The semantic encoding module 20 copies the current model and saves it as the old semantic encoding module 20 model And put it in the test state. The incremental update of the semantic encoding module 20 network will continue to train the network T from the parameter θ1 θ .

[0137] Step 3: The semantic decoding module 40 copies the feature recognition model and saves it as the original feature recognition model And put it in testing state.

[0138] Step 4: Update the network structure of the result prediction part of the semantic decoding module 40: increase the heat map prediction output category to the number of target categories that need to be incrementally updated in the new task, and initialize the parameters of this part of the network ψ newinit The incremental update of the semantic decoding module 40 network will be from {ψ1,ψ newinit}Continue training the network.

[0139] Step 5: Use dataset B to analyze SC θ,Ψ Perform incremental update training, with data flowing in parallel through the old network SC in the test state θ,Ψ And the new network in the training state. We pass the data through SC θ,Ψ The obtained heat map prediction result is recorded as Data is trained on SC θ,Ψ The obtained heat map prediction result is recorded as Results included on old categories and the results on the new categories Since the true category of dataset B should fall In and And the label of dataset B to find the target detection loss, denoted as L new (new category detection loss). We use and Find the MSE loss, denoted as L old .

[0140] It is worth noting that the output results of the current feature recognition network model 42 are increased with the same number of recognition results as the new categories, so that the current feature recognition network model 42 outputs the recognition results of the original categories and the recognition results of the new categories. Further, a loss function can be constructed with the recognition results of the original categories in the original feature recognition network model 41, thereby ensuring the accuracy of the original category recognition.

[0141] The parameter optimization module 50 is also used to construct the original category loss function according to the original model test feature recognition result and the current model test feature recognition result;

[0142] The parameter optimization module 50 is further configured to construct a new category loss function based on the new category recognition result of the current model and the new category label;

[0143] It is understandable that the feature label of the new category in the existing features to be identified can be an artificial annotation of the new category, which can be used to compare with the new category recognition result of the existing model to determine the accuracy of the new category recognition result of the existing model.

[0144] The parameter optimization module 50 is further configured to obtain a total loss function based on the original category loss function, the preset balance parameter, and the new category loss function;

[0145] It should be further explained that the new category loss function in the total loss function can be used to constrain the target detection accuracy of the target detection network model for the new category features; the original category loss function can be used to constrain the target detection accuracy of the target detection network model for the original category features; the total loss function constructed based on the new category loss function and the original category loss function can simultaneously constrain the accuracy of both. Figure 4 , the total loss function can refer to the following formula:

[0146] Ltotal=Lnew+λ*Lold

[0147] Among them, L total Represents the total loss function, L new Represents the new category loss function, L old Represents the original category loss function, and λ represents the balance parameter.

[0148] It should be noted that L oldThe original category loss function represented by can be the mean square error loss of the current model test feature recognition result and the original model test feature recognition result.

[0149] It is worth noting that when a new category needs to be identified, the new category identification result constitutes the new category loss function, and the test feature identification result constitutes the original category loss function. The new category loss function and the original category loss function together constitute the total loss function of the entire target detection network model. The target detection network model is constrained by the total loss function, so that the target detection network model can improve the recognition accuracy of new categories during the training process while ensuring the recognition accuracy of the original categories.

[0150] The parameter optimization module 50 is further used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

[0151] It can be understood that the total loss function optimizes the entire target detection network model, including the feature extraction neural network and the feature recognition network model.

[0152] It is understandable that the reference target detection network model can be a target detection network model after the target detection network model is optimized by the total loss function, and the reference target detection network model can accurately identify new category targets and original category targets in the data to be detected.

[0153] It is worth noting that when a new category needs to be identified, the new category identification result constitutes the new category loss function, and the test feature identification result constitutes the original category loss function. The new category loss function and the original category loss function together constitute the total loss function of the entire target detection network model. The target detection network model is constrained by the total loss function, so that the target detection network model can improve the recognition accuracy of new categories during the training process while ensuring the recognition accuracy of the original categories.

[0154] When new data arrives, we use it to incrementally update and train the end-to-end object detection network model (which can be a complete object detection model consisting of a feature extraction network model and a feature recognition network model). This allows the new model to accurately detect both old and new categories. New data is defined as objects in images of new categories that the old model cannot accurately detect.

[0155] After copying the feature recognition network model, this embodiment performs feature recognition on the features to be recognized, respectively, to obtain feature recognition results containing new categories and original categories, and feature recognition results containing only original categories. Further, based on the recognition results of the two original categories and the recognition results of the new category, a total loss function is constructed that simultaneously constrains the recognition accuracy of the new category and the recognition accuracy of the original category. According to the total loss function, the overall target detection network model is optimized to obtain a target detection network model, and accurate recognition of incremental tasks including new categories and original categories is achieved through the target detection network model.

[0156] The embodiment of the present invention provides a target detection semantic communication system for incremental tasks, referring to Figure 8 , Figure 8 This is a system structure diagram of the third embodiment of an incremental task-oriented target detection semantic communication system of the present invention.

[0157] The semantic encoding module 20 is further configured to, when the state signal is a first test state signal, initialize parameters of the feature extraction network model based on the first test state signal to obtain an initial feature extraction network model;

[0158] The semantic encoding module 20 is further configured to input the image dataset to be detected into the initial feature extraction network model to obtain initial features to be identified;

[0159] The semantic encoding module 20 is further configured to send the initial features to be identified and the status signal to the semantic decoding module 40 via the wireless channel 30 .

[0160] The semantic decoding module 40 is further configured to, upon receiving the initial feature to be identified and the initial test status signal, initialize the feature recognition network model based on the initial test status signal to obtain an initial feature recognition network model;

[0161] It is understandable that the first test status signal may represent that the feature recognition network model is performing target detection for the first time, or it may be a signal that the task category list is cleared and the feature recognition network model is retrained.

[0162] It should be understood that when the first test status signal is received, regardless of whether the parameters of the feature recognition network model have been initialized at this time, the feature recognition network model is reinitialized to ensure that the feature recognition network model is detecting the target for the first time.

[0163] It can be understood that the neural network model obtained after initializing the parameters of the feature recognition network model is called the initial feature extraction network model.

[0164] The semantic decoding module 40 is further configured to input the initial features to be identified into the initial feature recognition model to obtain an initial target detection result.

[0165] It is understandable that the initial features to be identified may be feature data obtained after feature extraction by the initial feature extraction neural network of the semantic encoding module 20 .

[0166] It should be understood that the initial recognition result can be obtained by using an initial feature extraction network model to identify the initial features to be identified based on the artificially set category to be identified.

[0167] The parameter optimization module 50 is further configured to construct an initial loss function based on the initial target detection result and the category label of the initial feature to be identified;

[0168] It is understandable that the category label of the initial feature to be identified is a mark that is artificially made in advance for the category to be identified in the image dataset to be detected.

[0169] In a specific implementation, the current initial features to be identified are 100 pictures, and the categories to be identified are cars and houses. The cars and houses in the 100 pictures are manually circled in the form of squares. The manually circled cars and houses can be understood as the category labels of the initial features to be identified.

[0170] It should be understood that the initial loss function can be a function of the difference between the manually labeled category labels and the categories recognized by the target detection network model.

[0171] The parameter optimization module 50 is further configured to perform parameter optimization on the feature extraction network model and the feature recognition network model according to the initial loss function to complete target detection for the first task.

[0172] It is understandable that the initial loss function optimizes the target detection network model by adjusting the parameters in the target detection network model through the initial loss function, so that the feature recognition results have a higher degree of overlap with the manually marked category labels.

[0173] This embodiment initializes the parameters of the feature extraction network model when receiving the first test status signal, restarts training the feature extraction neural network, identifies new categories in the feature data, and achieves accurate recognition of new categories. The user can also flexibly adjust the neural network by changing the task category list.

[0174] The embodiment of the present invention provides a target detection semantic communication system for incremental tasks, referring to Figure 9 , Figure 9 This is a system structure diagram of the third embodiment of an incremental task-oriented target detection semantic communication system of the present invention.

[0175] The semantic encoding module 20 is further configured to input the image dataset to be detected into the feature extraction network model 24 based on the test status signal when receiving the test status signal to obtain the test features to be identified;

[0176] The semantic encoding module 20 is further configured to send the test feature to be identified and the test status signal to the semantic decoding module 40 via the wireless channel 30 .

[0177] It should be noted that, in a specific implementation, assuming that the input is an image x and the output is a semantic feature matrix y, the neural network can be expressed by the following formula:

[0178] y=Tθ(x)

[0179] Among them, T θ (·) represents the mapping function of the feature extraction network model of the semantic encoding module 20.

[0180] The semantic decoding module 40 is further configured to, upon receiving the test feature to be identified and the test status signal, input the training feature to be identified into the feature recognition network model 44 based on the test status signal to obtain a test feature recognition result.

[0181] It is understandable that the test state signal may represent that there are no new categories to be identified at this time, and the training data to be identified may be directly input into the feature recognition network model.

[0182] It should be understood that what needs to be identified are the original categories, and the test feature recognition results are obtained through the feature recognition network model.

[0183] The parameter optimization module 50 is further configured to construct a test loss function based on the test feature recognition result and the category label of the training feature to be recognized;

[0184] It is understandable that the original category labels in the to-be-identified training feature data may be the to-be-identified categories that are artificially circled in the to-be-identified data set.

[0185] The parameter optimization module 50 is further used to optimize the parameters of the feature extraction network model and the feature recognition network model 44 according to the test loss function to complete the target detection of the test task.

[0186] It is understandable that at this time, only the original category is to be identified. The loss function of the original category is established based on the identification status of the original category in the feature to be identified and the manually marked original category label. The parameters of the feature recognition network model 44 are adjusted through the loss function of the original category, and the feature recognition network model 44 is optimized and trained, so as to more accurately detect the original category in the data set to be identified.

[0187] It should be emphasized that the feature to be identified includes at least one of the original feature to be identified, the current feature to be identified, the initial feature to be identified, and the test feature to be identified;

[0188] The semantic encoding module 20 is further configured to compress the feature to be identified to obtain a compressed feature to be identified, and send the compressed feature to be identified to the semantic decoding module 40 via the wireless channel 30;

[0189] The semantic decoding module 40 is further configured to receive the compressed features to be identified, and perform feature decoding on the compressed features to be identified to obtain the features to be identified.

[0190] In the specific implementation, take the VOC dataset as an example, which contains 20 categories of targets. Assuming that Task 1 is to detect the first 10 categories, we can use the image dataset A containing the first 10 categories of targets to train an SC θ,Ψ However, in reality, target detection tasks and data are not fixed, but fluid. Now the detection task or detection data changes, and new categories appear in the image. θ,Ψ It is impossible to detect it effectively. Assuming that Task 2 is the last 10 categories of the VOC dataset, we can use the image dataset B containing the last 10 categories of targets to detect SC θ,Ψ Update so that the new model can accurately detect both A and B.

[0191] Furthermore, drawing on the classic deep learning model CenterNet for target detection, the neural network structure design of the semantic encoding module 20 can be referenced Figure 10 , which consists of two parts: feature extraction and feature compression. The feature extraction network comes from the ResNet50 network of CenterNet, which is not introduced in this patent. The feature extraction module inputs a 512x512x3 image and outputs 16x16x2048 features. We add a feature compression module after it, which consists of a convolutional layer, a batch normalization layer, and a nonlinear activation layer. Assume that the feature dimension after compression is m( Figure 5 Where m=16), the dimension of the feature y output by the semantic encoding module 20 is 16x16xm.

[0192] The semantic decoding module 40 neural network structure design can be referenced Figure 11 , which consists of three parts: feature recovery, feature decoding and result prediction. The feature decoding and result prediction networks come from CenterNet and are not introduced in this patent. We add a feature recovery module before it, which consists of a deconvolution layer, a batch normalization layer and a nonlinear activation layer. The input of the feature recovery module is a 16x16xm Output features The dimensions are 16x16x2048. The target detection results are then output through the feature decoding and result prediction modules.

[0193] It is worth noting that the state signal and the number of new categories are obtained according to the list of categories to be detected and task categories, and different feature extraction neural networks and feature recognition network models are obtained according to different state signals and the number of new categories. The target detection network model is further optimized according to the recognition results of different feature recognition network models, so as to more accurately identify various types of data to be detected.

[0194] It should be further explained that the task category list is a blank list in the initial state. At this time, no matter what the current parameter values ​​of the feature extraction neural network are, the current parameters are initialized and the feature extraction neural network training is restarted. The user can flexibly adjust the neural network by changing the task category list.

[0195] It is understandable that when there is no new category to be identified, the parameters of the current feature extraction network model 22 are retained, feature extraction is performed directly, and a loss function is constructed based on the recognition results obtained by identifying the features to achieve repeated training of existing category target detection and improve the accuracy of target detection.

[0196] In this embodiment, upon receiving an object recognition task, the semantic encoding module 20 performs feature extraction and sends only the feature data to the semantic decoding module 40. This reduces the amount of data processing required by the semantic decoding module 40 and conserves communication bandwidth. When a new category task is added, the feature extraction neural network is replicated, and the feature data obtained by each feature extraction neural network is sent to the corresponding feature recognition network, providing a basis for subsequent neural network parameter updates.

[0197] In this embodiment, when the semantic encoding module receives an object recognition task, it performs feature extraction in the semantic encoding module and sends only the feature data to the semantic decoding module, reducing the data processing workload of the semantic decoding module and conserving communication bandwidth. When a new category task is added, the feature extraction neural network is replicated, and the feature data obtained by each feature extraction neural network is sent to the corresponding feature recognition network, providing a basis for subsequent neural network parameter updates.

[0198] In addition, reference Figure 12 Based on the first embodiment, the present invention further proposes a target detection semantic communication method for incremental tasks, characterized in that the target detection semantic communication method for incremental tasks is applied to the target detection semantic communication system for incremental tasks, and the method includes:

[0199] The task preprocessing module obtains a dataset of images to be detected and categories to be detected, determines the number of new categories according to the categories to be detected, obtains a status signal according to the number of new categories, and sends the status signal and the dataset of images to be detected to the semantic encoding module;

[0200] The semantic encoding module receives the status signal, and when the status signal is an incremental update status signal, performs feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and sends the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel;

[0201] The semantic decoding module receives the original feature to be identified, the current feature to be identified, and the incremental update status signal, and performs feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result;

[0202] The parameter optimization module constructs a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimizes the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the incremental update target detection training.

[0203] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any limitation on this.

[0204] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0205] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0206] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0207] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0208] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target detection semantic communication system for incremental tasks, characterized by: The incremental task-oriented target detection semantic communication system includes a task preprocessing module, a semantic encoding module, a wireless channel, a parameter optimization module and a semantic decoding module; The task preprocessing module is used to obtain a dataset of images to be detected and categories to be detected, determine the number of new categories based on the categories to be detected, obtain a status signal based on the number of new categories, and send the status signal and the dataset of images to be detected to the semantic encoding module; The task preprocessing module is further configured to obtain a task category list, determine whether the task category list is a blank list, and obtain a first training state signal if the task category list is a blank list; The task preprocessing module is further configured to, when the task category list is not a blank list, traverse the task category list and match the category to be detected with the task category in the task category list; The task preprocessing module is further configured to obtain a test status signal when the category to be detected successfully matches all the task categories in the task category list; The task preprocessing module is further configured to, when a match fails between the category to be detected and the task category in the task category list, obtain a new number of categories based on the number of failed matches between the category to be detected and the task category in the task category list, and obtain an incremental update status signal based on the new number of categories; The semantic encoding module is configured to receive the status signal and, when the status signal is an incremental update status signal, perform feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and transmit the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel; The semantic decoding module is configured to receive the original feature to be identified, the current feature to be identified, and the incremental update status signal, and perform feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result; The parameter optimization module is used to construct a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

2. The incremental task-oriented object detection semantic communication system according to claim 1, characterized in that: The semantic encoding module is further configured to, upon receiving an incremental update status signal, copy the feature extraction network model based on the incremental update status signal to obtain an original feature extraction network model and a current feature extraction network model; The semantic encoding module is further used to input the image data set to be detected into the original feature extraction network model and the current feature extraction network model at the same time to obtain the original features to be identified and the current features to be identified; The semantic encoding module is further configured to send the original feature to be identified, the current feature to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel.

3. The incremental task-oriented object detection semantic communication system according to claim 2, characterized in that: The new category recognition result includes the new category recognition result of the current model, and the original category recognition result includes the original category recognition result of the model and the original category recognition result of the current model; The semantic decoding module is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model and the current feature recognition network model; The semantic decoding module is further used to input the original features to be identified into the original feature recognition network model to obtain the original model original category recognition result; The semantic decoding module is further used to input the current features to be identified into the current feature recognition network model to obtain the new category recognition result of the current model and the original category recognition result of the current model.

4. The incremental task-oriented object detection semantic communication system according to claim 3, characterized in that: The semantic decoding module is further configured to, upon receiving the original feature to be identified, the current feature to be identified, and the incremental update status signal, copy the feature recognition network model based on the incremental update status signal to obtain the original feature recognition network model and the initial current feature recognition network model; The semantic decoding module is further configured to add a corresponding number of new categories to the initial existing feature recognition network model according to the number of new categories in the incremental update state signal, wherein the recognition result of the initial existing feature recognition network model includes the original categories and the new categories; The semantic decoding module is further used to initialize the parameters corresponding to the newly added categories in the initial current feature recognition network model to obtain the current feature recognition network model.

5. The incremental task-oriented object detection semantic communication system according to claim 3, characterized in that: The parameter optimization module is further used to construct an original category loss function based on the original model original category recognition result and the current model original category recognition result; The parameter optimization module is further used to construct a new category loss function based on the new category recognition result of the current model and the new category label; The parameter optimization module is further configured to obtain a total loss function based on the original category loss function, the preset balance parameter, and the new category loss function; The parameter optimization module is also used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the target detection of the incremental task.

6. The incremental task-oriented object detection semantic communication system according to claim 1, characterized in that: The semantic encoding module is further configured to, when the state signal is a first training state signal, initialize parameters of the feature extraction network model based on the first training state signal to obtain an initial feature extraction network model; The semantic encoding module is further configured to input the image dataset to be detected into the initial feature extraction network model to obtain initial features to be identified; The semantic encoding module is further configured to send the initial features to be identified and the status signal to the semantic decoding module via a wireless channel.

7. The incremental task-oriented object detection semantic communication system according to claim 6, characterized in that: The semantic decoding module is further configured to, upon receiving the initial feature to be identified and the first training state signal, initialize the feature recognition network model based on the first training state signal to obtain an initial feature recognition network model; The semantic decoding module is further configured to input the initial features to be identified into the initial feature recognition model to obtain an initial target detection result.

8. The incremental task-oriented object detection semantic communication system according to claim 7, characterized in that: The parameter optimization module is further configured to construct an initial loss function based on the initial target detection result and the category label of the initial feature to be identified; The parameter optimization module is further used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the initial loss function to complete the target detection of the first task.

9. The incremental task-oriented object detection semantic communication system according to claim 1, characterized in that: The semantic encoding module is further configured to, upon receiving a test status signal, input the image data set to be detected into a feature extraction network model based on the test status signal to obtain a test feature to be identified; The semantic encoding module is further configured to send the test feature to be identified and the test status signal to the semantic decoding module via a wireless channel.

10. The incremental task-oriented object detection semantic communication system according to claim 9, characterized in that: The semantic decoding module is further configured to, upon receiving the test feature to be identified and the test status signal, input the training feature to be identified into a feature extraction network model based on the test status signal to obtain a test feature recognition result.

11. The incremental task-oriented object detection semantic communication system according to claim 10, characterized in that: The parameter optimization module is further configured to construct a test loss function based on the test feature recognition result and the category label of the training feature to be recognized; The parameter optimization module is further used to optimize the parameters of the feature extraction network model and the feature recognition network model according to the test loss function to complete the target detection of the test task.

12. The incremental task-oriented object detection semantic communication system according to any one of claims 1 to 11, characterized in that: The feature to be identified includes at least one of the original feature to be identified, the current feature to be identified, the initial feature to be identified, and the test feature to be identified; The semantic encoding module is further configured to compress the feature to be identified to obtain a compressed feature to be identified, and send the compressed feature to be identified to the semantic decoding module via a wireless channel; The semantic decoding module is further configured to receive the compressed features to be identified, and perform feature decoding on the compressed features to be identified to obtain the features to be identified.

13. A semantic communication method for target detection in incremental tasks, characterized in that: The incremental task-oriented target detection semantic communication method is applied to the incremental task-oriented target detection semantic communication system, and the method includes: The task preprocessing module obtains a dataset of images to be detected and categories to be detected, determines the number of new categories according to the categories to be detected, obtains a status signal according to the number of new categories, and sends the status signal and the dataset of images to be detected to the semantic encoding module; The task preprocessing module obtains a task category list, determines whether the task category list is a blank list, and obtains a first training state signal if the task category list is a blank list; When the task category list is not a blank list, the task preprocessing module traverses the task category list and matches the category to be detected with the task category in the task category list; The task preprocessing module obtains a test status signal when the category to be detected and the task categories in the task category list are all successfully matched; When there is a match failure between the category to be detected and the task category in the task category list, the task preprocessing module obtains a new category number according to the number of failed matches between the category to be detected and the task category in the task category list, and obtains an incremental update status signal according to the new category number; The semantic encoding module receives the status signal, and when the status signal is an incremental update status signal, performs feature extraction on the image data set to be detected based on the incremental update status signal to obtain original features to be identified and current features to be identified, and sends the original features to be identified, the current features to be identified, and the incremental update status signal to the semantic decoding module via a wireless channel; The semantic decoding module receives the original feature to be identified, the current feature to be identified, and the incremental update status signal, and performs feature recognition on the original feature to be identified and the current feature to be identified based on the incremental update status signal to obtain a new category recognition result and an original category recognition result; The parameter optimization module constructs a total loss function based on the new category recognition results, the original category recognition results and the new category labels of the image data set to be detected, and optimizes the parameters of the feature extraction network model and the feature recognition network model according to the total loss function to complete the incremental update target detection training.

Images