Method and system for semantic communication transmission of substation inspection images

Abstract

The application provides a substation inspection image semantic communication transmission method and system, the method comprises the following steps: inputting a target inspection image collected by itself into a preset semantic coding model in a substation, so that the semantic coding model outputs a semantic feature vector corresponding to the target inspection image; transmitting the semantic feature vector of the target inspection image to an image receiving end through a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain a transmission inspection image corresponding to the target inspection image, wherein the semantic coding model and the semantic decoding model are obtained by pre-training in a joint coding manner of a signal source channel. The application can transmit the inspection image under the condition of low signal-to-noise ratio and effectively improve the transmission quality of the inspection image.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a semantic communication transmission method and system for substation inspection images. Background Technology

[0002] Semantic communication is a novel architecture that integrates user needs and information meaning into the communication process, significantly improving transmission efficiency by exploring semantic information. In recent years, the continuous development of artificial intelligence technology, with its significant advancements in algorithms, computing power, and data, has enabled the precise and efficient extraction of semantic information from images, text, and speech, making semantic communication possible at the engineering level. During substation inspections, image acquisition terminals, such as inspection robots, may struggle to transmit inspection images properly when encountering weak signals or entering areas with low signal strength, thus reducing the quality of daily substation inspections.

[0003] Existing technologies primarily utilize traditional communication methods, namely, performing traditional image encoding on the image to be transmitted, then source encoding on the encoded result, transmitting it through a channel, and finally decoding it sequentially. Traditional image encoding is generally entropy encoding, which, by preserving as much information as possible, makes it difficult to achieve high information compression rates. Furthermore, traditional image transmission techniques utilize a combination of source encoding and channel encoding. This approach exhibits good characteristics at high signal-to-noise ratios (SNRs); however, as the SNR decreases, the image quality transmitted using this method deteriorates rapidly, and at extremely low SNRs, traditional transmission methods become ineffective.

[0004] Therefore, existing communication methods suffer from poor image transmission quality or difficulty in transmitting images under low signal-to-noise ratio conditions. Summary of the Invention

[0005] In view of this, embodiments of this application provide a semantic communication transmission method and system for substation inspection images to eliminate or improve one or more defects existing in the prior art.

[0006] The first aspect of this application provides a semantic communication transmission method for substation inspection images, the method comprising:

[0007] In the substation, the target inspection image collected by itself is input into a preset semantic coding model so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image;

[0008] The semantic feature vector of the target inspection image is transmitted to the image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method.

[0009] In some embodiments of this application, the semantic coding model includes: an encoder and a key semantic enhancement system;

[0010] Correspondingly, the step of inputting the target inspection image collected in the substation into a preset semantic coding model, so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image, includes:

[0011] In the substation, the target inspection image collected by itself is input into the key semantic enhancement system, so that the key semantic enhancement system can identify and extract key feature sub-images from the target inspection image based on the preset key feature recognition rules. Then, both the target inspection image and the key feature sub-image are input into the encoder for semantic encoding to obtain the semantic feature encoding vectors corresponding to the target inspection image and the key feature sub-image respectively. Based on the semantic feature encoding vectors corresponding to the key feature sub-images, the semantic feature encoding vectors corresponding to the target inspection image are semantically enhanced to obtain the semantic feature vectors corresponding to the target inspection image.

[0012] In some embodiments of this application, before inputting the target inspection image collected by the substation into a preset semantic coding model so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image, the method further includes:

[0013] Obtain a training set containing historical inspection images with semantically encoded labels;

[0014] A source-channel joint coding approach is adopted, and the training set is used to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic coding model for semantic coding of inspection images is stored locally.

[0015] In some embodiments of this application, the method of joint source-channel coding, using the training set to train a preset machine learning network architecture, to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of semantic feature vectors corresponding to inspection images, includes:

[0016] In offline mode, the target neural network of the key semantic enhancement system detects historical inspection images with semantic coding labels in the training set based on preset key feature recognition rules and cuts them to obtain the corresponding region images of the historical inspection images with semantic coding labels in the training set. The encoder encodes the historical inspection images with semantic coding labels and the corresponding region images in the training set to obtain the semantic codes of the historical inspection images with semantic coding labels and the corresponding region images. The semantic codes of the historical inspection images with semantic coding labels and the corresponding region images in the training set are quantized to obtain overall compression codes and region compression codes. A simulated noise vector is simulated according to the channel simulation model and superimposed on the overall compression code and the region compression code to obtain overall simulation codes and region simulation codes. The semantic enhancement system in the key semantic enhancement system enhances the overall simulation code with the region simulation code to obtain the semantic feature enhancement vector corresponding to the historical inspection images with semantic coding labels in the training set. Based on the training process of the target neural network, encoder, and semantic enhancement system, a semantic coding model for semantic coding of the inspection images is obtained.

[0017] In offline mode, the discriminator concatenates the overall simulation code and the region simulation code with historical inspection images labeled with semantic codes, respectively, and inputs them into a pre-trained VGG network to obtain overall floating-point numbers and region floating-point numbers, respectively. The overall floating-point numbers and the region floating-point numbers are then cross-entropy calculated with natural image label values ​​to obtain the overall adversarial generation loss function and the region adversarial generation loss function, respectively, to improve the image generation quality of the decoder. The decoder decodes the overall simulation code and the region simulation code to obtain the overall inspection image and the region inspection image, respectively. Based on the training process of the discriminator and the decoder, a semantic decoding model for semantic decoding of the inspection image is obtained.

[0018] In some embodiments of this application, the key semantic enhancement system in the semantic coding model includes:

[0019] A target neural detection network is used to detect the target inspection image based on preset key feature recognition rules and cut out the region image corresponding to the target inspection image;

[0020] A semantic enhancement system is used to semantically enhance the semantic feature encoding vector corresponding to the target inspection image based on the semantic feature encoding vector corresponding to the key feature sub-image to obtain the semantic feature vector corresponding to the target inspection image.

[0021] The encoder in the semantic coding model is used to encode the target inspection image to obtain the semantic feature vector corresponding to the target inspection image.

[0022] A second aspect of this application provides a semantic communication transmission method for substation inspection images, the method comprising:

[0023] The semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation is received via a wireless channel. The semantic feature vector corresponding to the target inspection image is output by the semantic coding model after the image acquisition terminal inputs the target inspection image it acquires in the substation into the preset semantic coding model.

[0024] The semantic feature vector of the target inspection image is input into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method.

[0025] In some embodiments of this application, before receiving the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via a wireless channel, the method further includes:

[0026] Obtain a training set containing historical inspection images with semantically encoded labels;

[0027] A source-channel joint coding approach is adopted, and the training set is used to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images is stored locally.

[0028] A third aspect of this application provides a semantic communication transmission system for substation inspection images, the system comprising:

[0029] Image acquisition and image receiving terminals based on wireless network communication connections;

[0030] The image acquisition terminal is used to execute the semantic communication transmission method for substation inspection images described in the first aspect above;

[0031] The image receiving end is used to execute the semantic communication transmission method for substation inspection images described in the second aspect above.

[0032] A fourth aspect of this application also provides an image acquisition terminal, comprising:

[0033] The image encoding module is used to input the target inspection image it collects in the substation into a preset semantic encoding model so that the semantic encoding model outputs the semantic feature vector corresponding to the target inspection image.

[0034] The image transmission module is used to transmit the semantic feature vector of the target inspection image to the image receiving end via a wireless channel, so that the image receiving end can input the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic encoding model and the semantic decoding model are trained in advance using a source-channel joint encoding method.

[0035] A fifth aspect of this application also provides an image receiving end, comprising:

[0036] The image encoding receiving module is used to receive the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via a wireless channel. The semantic feature vector corresponding to the target inspection image is output by the semantic encoding model after the image acquisition terminal inputs the target inspection image it acquires in the substation into a preset semantic encoding model.

[0037] The image generation module is used to input the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic coding model are trained in advance using a source-channel joint coding method.

[0038] The sixth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the semantic communication transmission method for substation inspection images described in the first aspect above, or implements the semantic communication transmission method for substation inspection images described in the second aspect above.

[0039] The seventh aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the semantic communication transmission method for substation inspection images described in the first aspect above, or implements the semantic communication transmission method for substation inspection images described in the second aspect above.

[0040] This application provides a semantic communication transmission method and system for substation inspection images. The method includes: inputting a target inspection image acquired in the substation into a preset semantic coding model, so that the semantic coding model outputs a semantic feature vector corresponding to the target inspection image; transmitting the semantic feature vector of the target inspection image to an image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain a transmitted inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are pre-trained using a source-channel joint coding method. This application can transmit inspection images under low signal-to-noise ratio conditions and effectively improve the transmission quality of inspection images.

[0041] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0042] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0044] Figure 1 This is a flowchart illustrating the semantic communication transmission method for a first substation inspection image in one embodiment of this application.

[0045] Figure 2 This is a flowchart illustrating the semantic communication transmission method for a second substation inspection image in another embodiment of this application.

[0046] Figure 3 This is a schematic diagram of the inspection image acquisition end in another embodiment of this application.

[0047] Figure 4 This is a schematic diagram of the structure of the inspection image receiving end in another embodiment of this application.

[0048] Figure 5This is an overall system architecture diagram of another embodiment of this application.

[0049] Figure 6 This is a basic system architecture diagram in another embodiment of this application.

[0050] Figure 7 This is a diagram of the key semantic enhancement system architecture in another embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0052] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0053] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0054] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0055] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0056] The following examples will provide a detailed description.

[0057] This application provides a semantic communication transmission method for first substation inspection images that can be executed by an image acquisition terminal. See [link to relevant documentation]. Figure 1 The semantic communication transmission method for the inspection images of the first substation specifically includes the following:

[0058] Step 110: Input the target inspection image collected by the substation into a preset semantic coding model so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image.

[0059] Step 120: The semantic feature vector of the target inspection image is transmitted to the image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method.

[0060] Specifically, in order to enable substations to transmit inspection images under low signal-to-noise ratio conditions, the image acquisition end (such as an inspection robot) inputs the target inspection image it acquires into a preset semantic coding model in the substation, so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image; then the semantic feature vector of the target inspection image is transmitted to the image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmitted inspection image corresponding to the target inspection image, thereby enabling the transmission of inspection images under low signal-to-noise ratio conditions.

[0061] The semantic coding model and semantic decoding model are trained in advance using source-channel joint coding. Simulated noise is added to the image transmission process through source-channel joint coding, thereby enhancing the transmission quality of the inspection image.

[0062] In order to improve the quality of transmitted inspection images under low signal-to-noise ratio conditions, in step 110, the semantic coding model includes: an encoder and a key semantic enhancement system;

[0063] Correspondingly, the step of inputting the target inspection image collected in the substation into a preset semantic coding model, so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image, includes:

[0064] Step 010: In the substation, the target inspection image collected by itself is input into the key semantic enhancement system, so that the key semantic enhancement system can identify and extract key feature sub-images from the target inspection image based on the preset key feature recognition rules. Then, both the target inspection image and the key feature sub-image are input into the encoder for semantic encoding to obtain the semantic feature encoding vectors corresponding to the target inspection image and the key feature sub-image respectively. Based on the semantic feature encoding vectors corresponding to the key feature sub-images, the semantic feature encoding vectors corresponding to the target inspection image are semantically enhanced to obtain the semantic feature vectors corresponding to the target inspection image.

[0065] Specifically, the image acquisition terminal in the substation inputs the target inspection image it acquires into the key semantic enhancement system. The key semantic enhancement system then identifies and extracts key feature sub-images from the target inspection image based on preset key feature recognition rules. Both the target inspection image and the key feature sub-images are then input into the encoder for semantic encoding to obtain the semantic feature encoding vectors corresponding to the target inspection image and the key feature sub-images, respectively. Based on the semantic feature encoding vectors corresponding to the key feature sub-images, the semantic feature encoding vectors corresponding to the target inspection image are semantically enhanced to obtain the semantic feature vectors corresponding to the target inspection image. This effectively improves the quality of transmitted inspection images under low signal-to-noise ratio conditions.

[0066] The key feature recognition rules are pre-defined by the inspection staff. For example, during the inspection process, the robot needs to collect data from dashboards. To collect images of these dashboards, staff need to pre-use images containing dashboards and manually mark their locations as data to train the robot's YOLO target neural network model. Following this process, the robot can then find and locate key semantic content regions.

[0067] To improve the effectiveness and reliability of model application, the following steps are included before step 110:

[0068] Step 011: Obtain a training set containing historical inspection images with semantically encoded labels.

[0069] Step 012: Using a source-channel joint coding method, the training set is applied to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of the inspection image and a semantic decoding model for semantic decoding of the semantic feature vector corresponding to the inspection image. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic coding model for semantic coding of the inspection image is stored locally.

[0070] Specifically, the image acquisition end obtains a training set containing historical inspection images with semantic coding labels; then, using source-channel joint coding, the training set is applied to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images, thereby improving the effectiveness and reliability of model application.

[0071] The machine learning network architecture includes a semantic coding model, a pre-defined channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic coding model is stored locally for semantic coding of the inspection images.

[0072] To further improve the effectiveness and reliability of model application, step 012 also includes:

[0073] Step 013: In offline mode, the target neural detection network of the key semantic enhancement system detects and cuts the historical inspection images with semantic coding labels in the training set according to the preset key feature recognition rules to obtain the region images corresponding to the historical inspection images with semantic coding labels in the training set; the encoder encodes the historical inspection images with semantic coding labels in the training set and the region images respectively to obtain the semantic codes of the historical inspection images with semantic coding labels in the training set and the semantic codes corresponding to the region images; the semantic codes of the historical inspection images with semantic coding labels in the training set and the semantic codes corresponding to the region images are quantized to obtain the overall compression code and the region compression code respectively; the channel simulation model simulates the noise vector and superimposes the simulated noise vector onto the overall compression code and the region compression code respectively to obtain the overall simulation code and the region simulation code; the semantic enhancement system in the key semantic enhancement system enhances the overall simulation code with the region simulation code to obtain the semantic feature enhancement vector corresponding to the historical inspection images with semantic coding labels in the training set; the semantic coding model for semantic coding of the inspection images is obtained according to the training process of the target neural detection network, encoder and semantic enhancement system.

[0074] Step 014: In offline mode, the discriminator concatenates the overall simulation code and the region simulation code with the historical inspection image with semantic coding labels, respectively, and inputs them into the pre-trained VGG network to obtain the overall floating-point number and the region floating-point number, respectively. The overall floating-point number and the region floating-point number are then cross-entropy calculated with the natural image label value to obtain the overall adversarial generation loss function and the region adversarial generation loss function to improve the image generation quality of the decoder. The decoder decodes the overall simulation code and the region simulation code to obtain the overall inspection image and the region inspection image, respectively. Based on the training process of the discriminator and the decoder, a semantic decoding model for semantic decoding of the inspection image is obtained.

[0075] See Figure 5Specifically, in the overall system architecture, the image acquisition end, in offline mode, uses the YOLO target neural network to detect and crop region images from historical inspection images with semantically encoded labels in the training set based on preset key feature recognition rules; the encoding process is described in [link to system architecture]. Figure 6 The basic system encodes historical inspection images and region images with semantic coding labels using an encoder to obtain semantic codes for the historical inspection images and region images respectively. The encoder is a convolutional neural network composed of several convolutional layers. Its input is an image (also considered as a vector in high-dimensional space), and its output is a semantic vector (a low-dimensional vector after feature extraction and dimensionality reduction by the convolutional layers). Let the computation process of the encoder network be a function E, the input image be denoted as x, and the output semantic vector be denoted as Y, then the encoding process is: Y = E(x). The semantic codes of the historical inspection images with semantic coding labels and the semantic codes corresponding to the region images in the training set are quantized to obtain overall compressed codes and region compressed codes respectively. A simulated noise vector is simulated according to the channel simulation model and superimposed on the overall compressed code and the region compressed code respectively to obtain overall simulated codes and region simulated codes. Specifically, let the quantized semantic code vector be Yq, and the semantic code vector after the channel simulation model be Yq. ', and the randomly generated noise vector is n (i.e., the randomly generated n mentioned in the previous note), then: Yq' = h*Yq + n, where h is the channel gain and n is the randomly generated noise vector; according to the semantic enhancement system in the key semantic enhancement system, the region simulation coding enhances the overall simulation coding to obtain the semantic feature vector corresponding to the historical inspection images with semantic coding labels in the training set. The specific operation is as follows: the semantic enhancement system inputs the overall simulation coding corresponding to the historical inspection images with semantic coding labels and the region simulation coding corresponding to the region images into the convolutional neural network, uses the region image part in the historical inspection images with semantic coding labels as the data label, and then uses the information that the region simulation coding has more than the data label to correct the overall simulation coding to obtain the semantic feature enhancement vector corresponding to the historical inspection images with semantic coding labels; finally, according to the training process of the target neural detection network, encoder and semantic enhancement system, a semantic coding model for semantic coding of inspection images is obtained, thereby further improving the effectiveness and reliability of the semantic coding model application.

[0076] Simultaneously, in offline mode, the image acquisition end uses the discriminator to concatenate the overall simulation code and the region simulation code with historical inspection images labeled with semantic codes, and inputs them into a pre-trained VGG network to obtain overall floating-point numbers and region floating-point numbers, respectively. The overall and region floating-point numbers are then cross-entropy calculated with natural image label values ​​to obtain the overall adversarial generation loss function and the region adversarial generation loss function, respectively, to improve the image generation quality of the decoder. A natural image label value of 1 indicates that the discriminator considers the image to be naturally captured, and 0 indicates that the discriminator considers the image to be generated. The decoder decodes the overall simulation code and the region simulation code to obtain the overall inspection image and the region inspection image, respectively. The decoder is a convolutional neural network composed of several residual blocks and upsampling convolutional networks. Its input is a semantic vector, and its output is an image. Inputting vectors into residual blocks (composed of a series of convolutional layers) does not change the vector size; the purpose of these convolutional layers is to fully learn and extract information from the semantic vector, enhancing the network's expressive power. Upsampling the input vector into a convolutional network alters the shape of the vector and increases its dimensionality. By setting appropriate upsampling kernel size and stride, we can match the input size of the entire decoder to the size of the semantic vector, while the output size is an image (HxWx3). Let the computation process of the decoder network be a function D, the input semantic vector be denoted as Yq', and the output image be X', then the encoding process is: X' = D(Yq'). Based on the training process of the discriminator and decoder, a semantic decoding model for semantic decoding of the inspected image is obtained, thereby further improving the effectiveness and reliability of the semantic decoding model application.

[0077] In the model training process, the inspection image refers to the historical inspection image with semantically encoded labels in the training set, and in the model application process, it refers to the target inspection image.

[0078] To further improve the quality of transmitted inspection images under low signal-to-noise ratio conditions, the key semantic enhancement system in step 010 includes:

[0079] A target neural detection network is used to detect the target inspection image based on preset key feature recognition rules and cut out the region image corresponding to the target inspection image;

[0080] A semantic enhancement system is used to semantically enhance the semantic feature encoding vector corresponding to the target inspection image based on the semantic feature encoding vector corresponding to the key feature sub-image to obtain the semantic feature vector corresponding to the target inspection image.

[0081] The encoder in the semantic coding model is used to encode the target inspection image to obtain the semantic feature vector corresponding to the target inspection image.

[0082] Specifically, see Figure 7 The target neural network uses the YOLO target neural network to detect and crop the target inspection image based on preset key feature recognition rules to obtain the corresponding region image of the target inspection image. The semantic enhancement system is used to semantically enhance the semantic feature encoding vector of the target inspection image according to the semantic feature encoding vector of the key feature sub-image to obtain the semantic feature vector of the target inspection image. The encoder in the semantic coding model is used to encode the target inspection image to obtain the semantic feature vector of the target inspection image, thereby further improving the quality of the transmitted inspection image under low signal-to-noise ratio conditions.

[0083] This application also provides a semantic communication transmission method for second substation inspection images that can be executed by an image receiving end, see [link to relevant documentation]. Figure 2 The semantic communication transmission method for the inspection images of the second substation specifically includes the following:

[0084] Step 210: Receive the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via a wireless channel. The semantic feature vector corresponding to the target inspection image is output by the semantic coding model after the image acquisition terminal inputs the target inspection image it acquires in the substation into a preset semantic coding model.

[0085] Step 220: Input the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image, wherein the semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method.

[0086] Specifically, the image receiving end (such as the substation client) receives the semantic feature vector corresponding to the target inspection image sent by the image acquisition end in the substation via a wireless channel. The semantic feature vector corresponding to the target inspection image is output by the semantic coding model after the image acquisition end inputs the target inspection image it acquires in the substation. Then, the semantic feature vector of the target inspection image is input into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image, thereby enabling the transmission of inspection images under low signal-to-noise ratio conditions.

[0087] The semantic coding model and semantic decoding model are trained in advance using a source-channel joint coding method, which can enhance the transmission quality of the inspection images.

[0088] To improve the effectiveness and reliability of model application, the following steps are included before step 210:

[0089] Step 211: Obtain a training set containing historical inspection images with semantically encoded labels.

[0090] Step 212: Using a source-channel joint coding method, the training set is applied to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of the inspection image and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection image. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection image is stored locally.

[0091] Specifically, the image receiving end acquires a training set containing historical inspection images with semantically coded labels; using a source-channel joint coding method, the training set is applied to train a preset machine learning network architecture to obtain a semantic coding model for semantically coding the inspection images and a semantic decoding model for semantically decoding the semantic feature vectors corresponding to the inspection images. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system; the semantic decoding model includes a decoder. The semantic decoding model for semantically decoding the semantic feature vectors corresponding to the inspection images is stored locally, thereby effectively improving the effectiveness and reliability of the semantic decoding model application.

[0092] This application also provides a semantic communication transmission system for substation inspection images, the system comprising:

[0093] Image acquisition and image receiving terminals are connected via wireless network communication.

[0094] The image acquisition terminal is used to execute the semantic communication transmission method for the first substation inspection image described above.

[0095] The image receiving end is used to execute the aforementioned semantic communication transmission method for the second substation inspection image.

[0096] This application also provides an embodiment of an image acquisition terminal for performing all or part of the semantic communication transmission method for the aforementioned first substation inspection image, see [link to embodiment]. Figure 3 The image acquisition terminal specifically includes the following components:

[0097] The image encoding module 10 is used to input the target inspection image it collects in the substation into a preset semantic encoding model so that the semantic encoding model outputs the semantic feature vector corresponding to the target inspection image.

[0098] The image transmission module 20 is used to transmit the semantic feature vector of the target inspection image to the image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic encoding model and the semantic decoding model are trained in advance using a source-channel joint encoding method.

[0099] The image acquisition terminal provided in this application can be used in the processing flow of the aforementioned semantic communication transmission method for the first substation inspection image. Its functions will not be repeated here, but can be found in the detailed description of the aforementioned semantic communication transmission method for the substation inspection image.

[0100] This application also provides an embodiment of an image receiving end for performing all or part of the aforementioned semantic communication transmission method for second substation inspection images, see [link to embodiment]. Figure 4 The image receiving end specifically includes the following:

[0101] The image encoding receiving module 30 is used to receive the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via a wireless channel. The semantic feature vector corresponding to the target inspection image is output by the semantic encoding model after the image acquisition terminal inputs the target inspection image it acquires in the substation into a preset semantic encoding model.

[0102] The image generation module 40 is used to input the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic coding model are trained in advance using a source-channel joint coding method.

[0103] The image receiving end provided in this application can be used to execute the processing flow of the aforementioned embodiment of the semantic communication transmission method for substation inspection images. Its functions will not be repeated here, but can be referred to the detailed description of the aforementioned embodiment of the semantic communication transmission method for substation inspection images.

[0104] This application provides a semantic communication transmission method and system for substation inspection images. The method includes: inputting a target inspection image acquired in the substation into a preset semantic coding model, so that the semantic coding model outputs a semantic feature vector corresponding to the target inspection image; transmitting the semantic feature vector of the target inspection image to an image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain a transmitted inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are pre-trained using a source-channel joint coding method. This application can transmit inspection images under low signal-to-noise ratio conditions and effectively improve the transmission quality of inspection images.

[0105] This application also provides an electronic device, such as a central server, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the semantic communication transmission method for substation inspection images mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.

[0106] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0107] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the semantic communication transmission method for substation inspection images in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the image classification model training method based on reinforced federated domain generalization in the above method embodiments.

[0108] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] The one or more modules are stored in the memory, and when executed by the processor, they execute the semantic communication transmission method for substation inspection images in the embodiment.

[0110] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0111] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0112] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0113] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned semantic communication transmission method for substation inspection images. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0114] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0115] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0116] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0117] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A semantic communication transmission method for substation inspection images, characterized in that, include: In the substation, the target inspection image collected by itself is input into a preset semantic coding model so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image; The semantic feature vector of the target inspection image is transmitted to the image receiving end via a wireless channel, so that the image receiving end inputs the semantic feature vector of the target inspection image into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method. The semantic encoding model includes: an encoder and a key semantic enhancement system; Correspondingly, the step of inputting the target inspection image collected in the substation into a preset semantic coding model, so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image, includes: In the substation, the target inspection image collected by itself is input into the key semantic enhancement system, so that the key semantic enhancement system can identify and extract key feature sub-images from the target inspection image based on the preset key feature recognition rules. Then, both the target inspection image and the key feature sub-image are input into the encoder for semantic encoding to obtain the semantic feature encoding vectors corresponding to the target inspection image and the key feature sub-image respectively. Based on the semantic feature encoding vectors corresponding to the key feature sub-images, the semantic feature encoding vectors corresponding to the target inspection image are semantically enhanced to obtain the semantic feature vectors corresponding to the target inspection image.

2. The semantic communication transmission method for substation inspection images according to claim 1, characterized in that, Before inputting the target inspection image collected in the substation into a preset semantic coding model so that the semantic coding model outputs the semantic feature vector corresponding to the target inspection image, the method further includes: Obtain a training set containing historical inspection images with semantically encoded labels; A source-channel joint coding approach is adopted, and the training set is used to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic coding model for semantic coding of inspection images is stored locally.

3. The semantic communication transmission method for substation inspection images according to claim 2, characterized in that, The method employs joint source-channel coding, using the training set to train a pre-defined machine learning network architecture to obtain a semantic coding model for semantically coding the inspection image and a semantic decoding model for semantically decoding the semantic feature vectors corresponding to the inspection image, including: In offline mode, the target neural network of the key semantic enhancement system detects historical inspection images with semantic coding labels in the training set based on preset key feature recognition rules and cuts them to obtain the corresponding region images of the historical inspection images with semantic coding labels in the training set. The encoder encodes the historical inspection images with semantic coding labels and the corresponding region images in the training set to obtain the semantic codes of the historical inspection images with semantic coding labels and the semantic codes of the corresponding region images. The semantic codes of the historical inspection images with semantic coding labels and the corresponding region images in the training set are quantized to obtain overall compression codes and region compression codes. A simulated noise vector is simulated according to the channel simulation model and superimposed on the overall compression code and the region compression code to obtain overall simulation codes and region simulation codes. The semantic enhancement system in the key semantic enhancement system enhances the overall simulation code with the region simulation code to obtain the semantic feature enhancement vector corresponding to the historical inspection images with semantic coding labels in the training set. Based on the training process of the target neural network, encoder, and semantic enhancement system, a semantic coding model for semantic coding of the inspection images is obtained. In offline mode, the discriminator concatenates the overall simulation code and the region simulation code with historical inspection images labeled with semantic codes, respectively, and inputs them into a pre-trained VGG network to obtain overall floating-point numbers and region floating-point numbers, respectively. The overall floating-point numbers and the region floating-point numbers are then cross-entropy calculated with natural image label values ​​to obtain the overall adversarial generation loss function and the region adversarial generation loss function, respectively, to improve the image generation quality of the decoder. The decoder decodes the overall simulation code and the region simulation code to obtain the overall inspection image and the region inspection image, respectively. Based on the training process of the discriminator and the decoder, a semantic decoding model for semantic decoding of the inspection image is obtained.

4. The semantic communication transmission method for substation inspection images according to claim 1, characterized in that, The key semantic enhancement systems in the semantic coding model include: A target neural detection network is used to detect the target inspection image based on preset key feature recognition rules and cut out the region image corresponding to the target inspection image; A semantic enhancement system is used to semantically enhance the semantic feature encoding vector corresponding to the target inspection image based on the semantic feature encoding vector corresponding to the key feature sub-image to obtain the semantic feature vector corresponding to the target inspection image. The encoder in the semantic coding model is used to encode the target inspection image to obtain the semantic feature vector corresponding to the target inspection image.

5. A semantic communication transmission method for substation inspection images, characterized in that, include: The system receives the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via a wireless channel. The semantic feature vector corresponding to the target inspection image is obtained by the image acquisition terminal in the substation inputting the target inspection image it has acquired into the key semantic enhancement system in the preset semantic coding model. Based on the preset key feature recognition rules, the system identifies and extracts key feature sub-images from the target inspection image. Then, the target inspection image and the key feature sub-images are both input into the encoder in the semantic coding model for semantic coding to obtain the semantic feature coding vectors corresponding to the target inspection image and the key feature sub-images, respectively. The semantic feature coding vectors corresponding to the target inspection image are semantically enhanced based on the semantic feature coding vectors corresponding to the key feature sub-images and then output. The semantic feature vector of the target inspection image is input into a preset semantic decoding model to obtain the transmission inspection image corresponding to the target inspection image. The semantic coding model and the semantic decoding model are trained in advance using a source-channel joint coding method.

6. The semantic communication transmission method for substation inspection images according to claim 5, characterized in that, Before receiving the semantic feature vector corresponding to the target inspection image sent by the image acquisition terminal in the substation via the wireless channel, the following is also included: Obtain a training set containing historical inspection images with semantically encoded labels; A source-channel joint coding approach is adopted, and the training set is used to train a preset machine learning network architecture to obtain a semantic coding model for semantic coding of inspection images and a semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images. The machine learning network architecture includes a semantic coding model, a preset channel simulation model, a discriminator, and a semantic decoding model connected in sequence. The semantic coding model includes an encoder and a key semantic enhancement system. The semantic decoding model includes a decoder. The semantic decoding model for semantic decoding of the semantic feature vectors corresponding to the inspection images is stored locally.

7. A semantic communication transmission system for substation inspection images, characterized in that, include: Image acquisition and image receiving terminals based on wireless network communication connections; The image acquisition terminal is used to execute the semantic communication transmission method for substation inspection images as described in any one of claims 1 to 4; The image receiving end is used to execute the semantic communication transmission method for substation inspection images as described in claim 5 or 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the semantic communication transmission method for substation inspection images as described in any one of claims 1-4, or implements the semantic communication transmission method for substation inspection images as described in any one of claims 5-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the semantic communication transmission method for substation inspection images as described in any one of claims 1-4, or implements the semantic communication transmission method for substation inspection images as described in any one of claims 5-6.

Images