Intelligent environment sensing method, system and device based on pipeline coding and multi-branch decoding

Abstract

The invention discloses an environment intelligent sensing method, system and device based on assembly line coding and multi-branch decoding, and particularly relates to the technical field of environment intelligent sensing, and the method comprises the steps: obtaining the monitoring image data of a plurality of sensors; processing the monitoring image data of each sensor, and performing identification based on a risk cognition model according to the processed monitoring image data of the sensors to obtain a multi-aspect risk visual identification result in the working environment of the power grid; and according to a multi-aspect risk visual identification result in the power grid working environment, correspondingly generating early warning information of various risk scenes, and transmitting the early warning information through a communication channel for feedback. According to the method, the data is divided into different stages through a pipeline coding technology, so that the data processing efficiency and throughput are improved. Meanwhile, parallel processing and comprehensive analysis of multi-source data are achieved through the multi-branch decoding technology, and therefore the environment intelligent sensing system can sense and understand environment information more comprehensively and more timely.

Description

Technical Field

[0001] The present invention relates to an environment intelligent perception method, system and device based on pipeline coding and multi-branch decoding, belonging to the technical field of environment intelligent perception. Background Art

[0002] Traditional environmental intelligent perception methods are subject to many limitations, including reliance on a single data source and lack of real-time performance. Currently, many environmental intelligent perception systems rely mainly on manual inspections and traditional sensor technologies, which usually only provide limited environmental data and often fail to achieve timely problem identification and response. In addition, traditional methods also face the challenge of low data processing and analysis efficiency, and cannot effectively cope with the growing amount and complexity of data.

[0003] In terms of data processing, although traditional encoding and decoding technologies can improve data transmission efficiency to a certain extent, their limitation lies in the failure to fully utilize multi-source data for comprehensive analysis and cognition. Traditional single encoding and decoding methods are often unable to cope with the complex changes in the environment and the growing demand for data processing, which leads to limitations in the performance of environmental intelligent perception. Traditional technologies can usually only process specific types of data, and there is a large time-consuming and energy-consuming process in the acquisition, processing and analysis of data, which limits its application in real-time environmental intelligent perception. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an intelligent environmental perception method, system and device based on pipeline coding and multi-branch decoding, which divides data into different stages through pipeline coding technology to improve the efficiency and throughput of data processing. At the same time, multi-branch decoding technology is used to achieve parallel processing and comprehensive analysis of multi-source data, so that the intelligent environmental perception system can perceive and understand environmental information more comprehensively and timely.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions: In a first aspect, the present invention provides an environment intelligent perception method based on pipeline coding and multi-branch decoding, comprising: acquiring multiple sensor monitoring image data; Processing each sensor monitoring image data to obtain processed sensor monitoring image data; According to the processed sensor monitoring image data, the risk recognition model is used to obtain the visual recognition results of various risks in the power grid working environment.

[0006] Furthermore, the sensor includes at least one of a temperature sensor, a humidity sensor, a photosensor and a vibration sensor, wherein the temperature sensor is used to monitor the temperature of the working environment of the power grid, the humidity sensor is used to monitor the humidity of the working environment of the power grid, the photosensor is used to monitor the light intensity of the working environment of the power grid, and the vibration sensor is used to monitor the vibration of the power grid equipment.

[0007] Further, the monitoring image data of each sensor is processed to obtain the processed sensor monitoring image data, including: integrating, denoising and correcting the monitoring image data of each sensor to obtain the processed sensor monitoring image data.

[0008] Furthermore, the risk perception model adopts an encoder-decoder structure; The encoder is used to perform a down-sampling operation on the processed sensor monitoring image data, which sequentially includes a basic convolution module and a plurality of feature encoding modules; The decoder includes a main branch and multiple sub-branches, the output ends of the main branch and the sub-branches are connected to the detection module through the fusion module, the input end of the main branch is connected to the output end of the encoder, and multiple feature decoding modules are arranged on it, the input end of the feature decoding module is connected to the corresponding basic convolution module and the feature encoding module, and the output end is connected to the corresponding sub-branch, and the sub-branch is provided with a feature decoding module; The basic convolution module and the feature encoding module are used to downsample the sensor monitoring image data after processing, and the feature decoding module is used to upsample the high-level semantic information output by the encoder and the shallow detail information output by the basic convolution module and the feature encoding module.

[0009] Furthermore, the feature encoding module includes a P-Block encoder, a normalization layer, and an activation function layer in sequence; The P-Block encoder includes a first convolutional layer, four intermediate convolutional layers and a second convolutional layer connected in sequence, the convolution kernel of the first convolutional layer is 1×1, which is used to expand the number of feature channels of the input image data, the four intermediate convolutional layers are 3×3 dilated convolutions with different dilation rates, which are used to reduce the output image data features to one quarter of the input image features, the convolution kernel of the second convolutional layer is 1×1, which is used to perform feature fusion on the output image data of the spliced ​​first convolutional layer and the four intermediate convolutional layers, and the activation function layer is a RELU activation function; The operation expression of the P-Block encoder is as follows: ; in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

[0010] Furthermore, the number of the sub-branches, feature encoding modules and feature decoding modules is 4, and the input end of the feature decoding module is respectively connected to the common convolution module and the first three feature encoding modules; The feature decoding module includes a fully connected layer, an upsampling module, a convolution layer and a batch normalization layer in sequence. The upsampling module is used to upsample the input image data using a bilinear interpolation method. The convolution kernel of the convolution layer is 1×1, which is used to change the number of image data channels to 2. The operation expression of the sub-branch is as follows: ; in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ; in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

[0011] Furthermore, an environment intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module; The environment intelligent perception enhancement module is used to enhance the environment detection part in the shallow detail information, which is divided into a first part and a second part; The first part includes a maximum pooling layer and an average pooling layer in parallel, the maximum pooling layer and the average convolution layer are both connected to a convolution layer, the input image feature data are fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and are dot-multiplied with the input image feature data to obtain the first part of output image feature data; The second part uses average pooling to collect contextual information with long dependencies in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts the image features through the convolution layer and then aggregates them through the fully connected layer, and then passes through the 1×1 convolution layer and the Sigmoid function layer in sequence to perform dot multiplication with the output image feature data of the first part to obtain the output image feature data.

[0012] Furthermore, the fusion module includes a fully connected layer and a 1×1 convolutional layer, and its operation expression is as follows: ; in, represents the output of the fusion module, express The convolution operation of size, Represents a concatenation operation.

[0013] Furthermore, it also includes a convolutional attention module, which is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence; The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain output image feature data; The operation flow of the channel attention module is expressed as follows: ; in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, represents the spatial attention weight, represents the output of the channel attention module; The operation flow of the spatial attention module is expressed as follows: ; in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

[0014] In a second aspect, the present invention provides an environment intelligent perception system based on pipeline coding and multi-branch decoding, the system is used to implement any of the above-mentioned environment intelligent perception methods based on pipeline coding and multi-branch decoding, including: A sensor data acquisition module is used to acquire multiple sensor monitoring image data; A data processing module is used to process the monitoring image data of each sensor to obtain processed sensor monitoring image data; The risk identification module is used to obtain visual identification results of various risks in the power grid working environment based on the processed sensor monitoring image data and the risk recognition model.

[0015] Furthermore, the data processing module is specifically used to integrate, denoise and correct the image data monitored by each sensor to obtain processed sensor monitoring image data.

[0016] Furthermore, the risk perception model adopts an encoder-decoder structure; The encoder is used to perform a down-sampling operation on the processed sensor monitoring image data, which sequentially includes a basic convolution module and a plurality of feature encoding modules; The decoder includes a main branch and multiple sub-branches, the output ends of the main branch and the sub-branches are connected to the detection module through the fusion module, the input end of the main branch is connected to the output end of the encoder, and multiple feature decoding modules are arranged on it, the input end of the feature decoding module is connected to the corresponding basic convolution module and the feature encoding module, and the output end is connected to the corresponding sub-branch, and the sub-branch is provided with a feature decoding module; The basic convolution module and the feature encoding module are used to downsample the sensor monitoring image data after processing, and the feature decoding module is used to upsample the high-level semantic information output by the encoder and the shallow detail information output by the basic convolution module and the feature encoding module.

[0017] Furthermore, the feature encoding module includes a P-Block encoder, a normalization layer, and an activation function layer in sequence; The P-Block encoder includes a first convolutional layer, four intermediate convolutional layers and a second convolutional layer connected in sequence, the convolution kernel of the first convolutional layer is 1×1, which is used to expand the number of feature channels of the input image data, the four intermediate convolutional layers are 3×3 dilated convolutions with different dilation rates, which are used to reduce the output image data features to one quarter of the input image features, the convolution kernel of the second convolutional layer is 1×1, which is used to perform feature fusion on the output image data of the spliced ​​first convolutional layer and the four intermediate convolutional layers, and the activation function layer is a RELU activation function; The operation expression of the P-Block encoder is as follows: ; in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

[0018] Furthermore, the number of the sub-branches, feature encoding modules and feature decoding modules is 4, and the input end of the feature decoding module is respectively connected to the common convolution module and the first three feature encoding modules; The feature decoding module includes a fully connected layer, an upsampling module, a convolution layer and a batch normalization layer in sequence. The upsampling module is used to upsample the input image data using a bilinear interpolation method. The convolution kernel of the convolution layer is 1×1, which is used to change the number of image data channels to 2; The operation expression of the sub-branch is as follows: ; in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ; in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

[0019] Furthermore, an environment intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module; The environment intelligent perception enhancement module is used to enhance the environment detection part in the shallow detail information, which is divided into a first part and a second part; The first part includes a maximum pooling layer and an average pooling layer in parallel, the maximum pooling layer and the average convolution layer are both connected to a convolution layer, the input image feature data are fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and are dot-multiplied with the input image feature data to obtain the first part of output image feature data; The second part uses average pooling to collect contextual information with long dependencies in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts the image features through the convolution layer and then aggregates them through the fully connected layer, and then passes through the 1×1 convolution layer and the Sigmoid function layer in sequence to perform dot multiplication with the output image feature data of the first part to obtain the output image feature data.

[0020] Furthermore, the fusion module includes a fully connected layer and a 1×1 convolutional layer, and its operation expression is as follows: ; in, represents the output of the fusion module, express The convolution operation of size, Represents a concatenation operation.

[0021] Furthermore, it also includes a convolutional attention module, which is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence; The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain output image feature data; The operation flow of the channel attention module is expressed as follows: ; in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, represents the spatial attention weight, represents the output of the channel attention module; The operation flow of the spatial attention module is expressed as follows: ; in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

[0022] In a third aspect, the present invention provides a computer device, comprising: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of any of the above-mentioned environmental intelligent perception methods based on pipeline coding and multi-branch decoding.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The environmental intelligent perception method provided by the present invention is used to detect various risk factors in the process of power grid operation. By integrating various sensors and artificial intelligence technologies, it can monitor the safety equipment status of power grid operators and the surrounding environment in real time to accurately identify potential safety hazards and provide early warnings and suggestions, thereby reducing the risk of power grid operation and ensuring the safety of operators. The present invention divides data into different stages through pipeline coding technology to improve the efficiency and throughput of data processing. At the same time, multi-branch decoding technology is used to achieve parallel processing and comprehensive analysis of multi-source data, so that the environmental intelligent perception system can perceive and understand environmental information more comprehensively and timely, not only can it better cope with the dynamic changes of the environment, but also can more effectively process large-scale data, providing a new technical solution for environmental intelligent perception; In practical applications, the environmental intelligent perception system of the present invention can be widely used in the power industry to improve the safety and efficiency of power grid operations, reduce accident risks, and protect the safety of life and property. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A schematic diagram of the structure of a risk cognition model of an environment intelligent perception method based on pipeline coding and multi-branch decoding in an embodiment of the present invention; Figure 2 A schematic diagram of a top view of a P-Block encoder of an environment intelligent perception method based on pipeline coding and multi-branch decoding in an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an environment intelligent perception enhancement module of an environment intelligent perception method based on pipeline coding and multi-branch decoding in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a decoder of an environment intelligent perception method based on pipeline coding and multi-branch decoding in an embodiment of the present invention; Figure 5 This is a structural diagram of a convolutional attention module of an environment intelligent perception method based on pipeline coding and multi-branch decoding in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0026] Embodiment 1: The environment intelligent perception method based on pipeline coding and multi-branch decoding provided by the embodiment of the present invention includes: Acquire multiple sensor monitoring image data, the sensors include temperature sensors, humidity sensors, photosensors and vibration sensors, but are not limited to these. Any sensor that can be used to detect multiple risk factors in the power grid operation process should be acceptable.

[0027] The humidity sensor and the humidity sensor are used to monitor the temperature and humidity of the working environment of the power grid respectively, the light sensor is used to monitor the light intensity of the working environment of the power grid, and the vibration sensor is used to monitor the vibration of the power grid equipment.

[0028] The monitoring image data of each sensor is processed to obtain the processed sensor monitoring image data, specifically including: integrating and archiving, clarifying, denoising and correcting the monitoring image data of each sensor to obtain the processed sensor monitoring image data to ensure data quality and accuracy.

[0029] According to the processed sensor monitoring image data, the risk recognition model is used to obtain the visual recognition results of various risks in the power grid working environment.

[0030] In this embodiment, the risk recognition model is based on deep learning and machine vision technology, and its pre-training is to improve the recognition accuracy by training and optimizing different risk types in power grid operations. Precision (P), recall (R), and average precision (mAP) are selected as important indicators for evaluating the stability of the model, where precision refers to the proportion of actual positive samples among all samples predicted as positive by the model. It measures the accuracy of the model when predicting as positive. High precision means that the model has fewer misjudgments when identifying positive samples. Recall refers to the proportion of all actual positive samples correctly predicted as positive by the model. It measures the recognition ability of the model for positive samples. High recall means that the model can better capture positive samples and reduce missed judgments. Average precision is an indicator for comprehensively evaluating the performance of the model on multiple categories. It combines the precision and recall of different categories and calculates their average. mAP is usually used for the evaluation of tasks such as target detection and image classification. A higher mAP value indicates that the model has better comprehensive performance on different categories.

[0031] During the model training process, precision measures the accuracy of the model when predicting the positive class, recall measures the model's ability to identify positive samples, and average precision combines the precision and recall of different categories and calculates their average. The superior performance of the model in various indicators has been verified through a large amount of experimental data, which shows that the model has high accuracy, recognition ability and comprehensive performance in the task of power grid operation risk recognition.

[0032] like Figure 1 As shown, the specific structure of the risk perception model adopts an encoder-decoder structure, and the encoder includes a basic convolution module and multiple feature encoding modules in sequence.

[0033] In this embodiment, the number of feature encoding modules is 4, and the feature encoding modules include a P-Block encoder, a normalization layer, and an activation function layer in sequence, and the activation function layer is a RELU activation function.

[0034] like Figure 2As shown in the figure, to simplify the process, the convolution operation of each layer before the output of the P-Block encoder is omitted in the figure. The P-Block encoder includes the first convolution layer, four intermediate convolution layers and the second convolution layer connected in sequence. The convolution kernel of the first convolution layer is 1×1, which is used to expand the number of feature channels of the input image data to ensure that the subsequent dimensionality reduction operation can fully learn the information. The four intermediate convolution layers are 3×3 dilated convolutions with different dilation rates to expand the receptive field without losing image resolution and capture multi-scale information at the same time. The dilated convolution itself does not introduce additional parameters, which meets the model requirements. Four consecutive convolution operations are performed through the four intermediate convolution layers. Each convolution operation reduces the output image data features to one-fourth of the input image features to achieve a deeper network with the least parameters. To avoid information loss, the output of each intermediate convolution layer will be spliced ​​together to enrich the information between channels and further expand the receptive field of the model. Finally, the feature fusion is performed through the 1×1 second convolution layer, so that the network can adaptively learn the different scale features of each convolution stage.

[0035] To prevent network degradation, the P-Block encoder adopts the skip connection operation proposed in RestNet. The operation expression of the P-Block encoder is as follows: ; in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

[0036] The P-Block encoder aims to train narrower and deeper neural networks with fewer parameters while integrating rich multi-scale information. Compared with existing technologies such as STDCNet, the P-Block encoder attaches equal importance to shallow and deep information in semantic segmentation, believing that high-level semantic features help to accurately identify target objects, while shallow detail information helps to refine the segmentation results. To maintain information fairness, the design of the P-Block encoder ensures that the number of shallow and deep information channels in each layer is equal. Compared with technologies such as Res2Net, it avoids the problem of uneven information weight, thereby improving the performance and stability of the model.

[0037] The specific operation process of the encoder is as follows: first, the original image data (i.e., sensor monitoring image data) is downsampled to half its size through the basic convolution module, and the final encoder output is one thirty-second of the original image. The P-Block module builds a narrower and deeper network coding model with fewer parameters, while fully integrating the different levels of features of the target object at different stages.

[0038] Considering the overall computational complexity and parameter count of the model, a convolutional attention module (CBAM) is used at the end of the encoding phase to enhance the texture features of the target object. CBAM focuses on the texture features of the image, which is of great significance for environmental intelligent perception technology. In high-resolution images, the texture of environmental features is usually very clear, so this module is very effective in distinguishing environmental features from background information. The CBAM module generates separate attention maps along two different dimensions. Compared with the previous SENet that only focused on channels, CBAM can achieve better results.

[0039] like Figure 5 As shown, the convolutional attention module is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence. The output of the encoder will first pass through the channel attention module. The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain the output image feature data.

[0040] The operation flow of the channel attention module is expressed as follows: ; in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, which maps a one-dimensional feature vector to a new one-dimensional feature vector, represents the spatial attention weight, Represents the output of the channel attention module.

[0041] The entire channel attention mechanism aggregates features through average pooling and maximum pooling, compresses the feature map into a one-dimensional vector, and then sends it to a shared network for learning, multiplies and merges element by element, and finally generates an attention feature map, which is then multiplied with the input image feature data to obtain the final output.

[0042] The operation flow of the spatial attention module is expressed as follows: ; in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

[0043] The spatial attention mechanism compresses the channel and performs average pooling and maximum pooling at the channel level to highlight the texture features of each pixel.

[0044] A traditional single-branch decoder may cause the deep semantic information learned by the encoder to gradually weaken during the gradual upsampling process. Therefore, the decoder of the present invention extends an additional branch at each stage to enhance the deep semantic information and directly upsample it to the original image size. This upsampling operation at different stages helps the decoder to learn multi-scale features.

[0045] like Figure 4 As shown in the figure, the decoder consists of a main branch and multiple sub-branches. The main branch is the main processing path of the decoder, which contains a series of convolutional layers, upsampling operations, and feature fusion operations. The main path is responsible for gradually restoring the spatial resolution of the feature map while maintaining the integrity of the semantic information. The input of the main branch is the deep feature map from the encoder, which contains rich semantic information but has low spatial resolution. The output is the feature map after multiple convolution and connection operations, which gradually restores the spatial resolution to a size close to the original image.

[0046] Each sub-branch receives feature maps from the encoder at different stages of the decoder and processes them independently. The input of the sub-branch is the output feature maps of the encoder at different stages. These feature maps have higher spatial resolution but less semantic information than deep feature maps. The output is the feature maps after the above processing, which will be fused with the output of the main path and other branches.

[0047] In this embodiment, the number of sub-branches is 4, which are the first branch, the second branch, the third branch, and the fourth branch. The input end of the main branch is connected to the output end of the encoder, and the output ends of the main branch and the sub-branch are connected to the detection module through the fusion module.

[0048] The main branch is provided with a plurality of connected feature decoding modules. In this embodiment, the number of feature decoding modules is 4, and each of the secondary branches is provided with a feature decoding module. The input end of the feature decoding module on the main branch is connected to the corresponding basic convolution module and the first three feature encoding modules, and the output end is connected to the feature decoding module on the corresponding secondary branch.

[0049] The feature decoding module includes a fully connected layer, an upsampling module, a convolutional layer, and a batch normalization layer in sequence. The upsampling module is used to upsample the input image data to the original image size using a bilinear interpolation method to reduce information loss, and then a 1×1 convolutional layer is used to change the number of image data channels to 2. Finally, a batch normalization operation is added to prevent the network gradient from disappearing.

[0050] The operation expression of the secondary branch is as follows: ; in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ; in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

[0051] In the generation In the process, four feature map concatenation and convolution operations are required, and each operation is followed by batch normalization. The specific steps are as follows: The first concatenation and convolution operation: The input of the fifth main branch (i.e. the final output of the encoder) and the input of the fourth branch (i.e. the output of the third feature encoding module) are upsampled by the bilinear interpolation method to obtain the feature map, which is then concatenated in the channel dimension. The convolution layer performs feature transformation to change the number of image data channels to 2, and finally applies batch normalization to obtain the intermediate result .

[0052] Second concatenation and convolution operation: the intermediate result The input image data of the third branch (i.e., the output of the second feature encoding module) is upsampled by the bilinear interpolation method to obtain the feature map, which is then concatenated in the channel dimension. The convolution layer performs feature transformation to change the number of image data channels to 2, and finally applies batch normalization to obtain the intermediate result .

[0053] The third concatenation and convolution operation: the intermediate result The input image data of the second branch (i.e., the output of the first feature encoding module) is upsampled by the bilinear interpolation method to obtain the feature map, which is then concatenated in the channel dimension. The convolution layer performs feature transformation to change the number of image data channels to 2, and finally applies batch normalization to obtain the intermediate result .

[0054] The fourth concatenation and convolution operation: the intermediate result The input image data of the first branch (i.e., the output of the basic convolution module) is upsampled by the bilinear interpolation method to obtain the feature map, which is then concatenated in the channel dimension. The convolutional layer performs feature transformation to change the number of image data channels to 2, and finally applies batch normalization to obtain .

[0055] The fusion module includes a fully connected layer and The convolution layer has the following operation expression: ; in, represents the output of the fusion module, express The convolution operation of size, Represents a concatenation operation.

[0056] Specifically, the main path receives the lowest level feature map from the encoder, undergoes a series of upsampling, After convolution and batch normalization, we generate Each branch receives the intermediate layer feature map from the encoder, and after upsampling, After convolution and batch normalization, we generate Finally, all the stages Superposition is performed according to the channel dimension, that is, , and use a size of The convolution kernel Process the superimposed output to generate the final output This design allows the decoder to capture environmental features at different scales and improve the performance of environmental intelligent perception techniques by fusing these features.

[0057] An environmental intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module.

[0058] like Figure 3 The figure shows the overall structure of the Environmental Intelligent Perception Enhancement Module (EIEM), which is divided into a first part and a second part. The first part includes a maximum pooling layer and an average pooling layer in parallel. Both the maximum pooling layer and the average convolution layer are connected to a convolution layer. The input image feature data are fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and are dot-multiplied with the input image feature data to obtain the output image feature data of the first part.

[0059] The first part adopts a channel selection structure, which combines average pooling and max pooling to enhance the network's perception of relatively important channels at each stage and suppress irrelevant information channels. These two pooling operations extract the average and maximum values ​​in the feature map, which helps the network identify significant areas and overall trends in the feature map. After average pooling and max pooling, use The convolution kernel is used to further extract local features in the feature map after the pooling operation, enhance the network's responsiveness to specific features, and provide richer information for subsequent fusion operations. After the convolution layer, the network uses the Add operation to add the results of average pooling and maximum pooling, which helps to merge the feature information extracted by the two pooling operations. Then, the weighted sum of the two features is combined with the output of the convolution layer through the dot product operation. The dot product operation can be regarded as a feature fusion technique that allows the network to learn the interactions and weights between different features, thereby obtaining a more comprehensive feature representation.

[0060] The second part uses average pooling to collect long-term dependent context information in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts image features through the convolution layer and aggregates them through the fully connected layer, and then passes through the The convolution layer, the Sigmoid function layer and the first part of the output image feature data are multiplied to obtain the output image feature data.

[0061] The second part mainly considers the spatial relationship between pixels. It draws on the idea of ​​cross attention and uses rectangular pooling kernels to establish long-term dependencies between discrete regions, rather than traditional square convolution kernels and pooling kernels. To further simplify the structure, average pooling is used to collect contextual information with long-term dependencies in both horizontal and vertical dimensions. The contextual information in the two dimensions passes through a convolution layer respectively, with the horizontal direction using The convolution kernel of the vertical direction is The main function of the two convolutional layers is to learn the feature interactions between different regions, so as to obtain richer feature representations. Then, these two dimensions are aggregated and The convolution operation is performed to learn the weight information between each direction. Finally, it is normalized by the Sigmoid function and multiplied with the output of the first stage to obtain the final output. Compared with the global average pooling, this rectangular pooling operation pays more attention to the long and narrow range rather than the entire feature map, thereby avoiding irrelevant calculations between some distant positions. Considering that most targets in environmental intelligent perception are continuous and internally consistent, this rectangular pooling kernel learning is suitable for these long and narrow shapes. In addition, compared with the self-attention module that requires a lot of calculations of the position relationship between each pixel, this module is more lightweight.

[0062] The above is the overall structure of the risk perception model.

[0063] Finally, based on the visual identification results of various risks in the power grid working environment, early warning information for various risk scenarios is generated. The early warning information for risk scenarios includes not wearing a safety belt, not wearing a safety helmet, hardware falling off, insulator defects, etc., but is not limited to these. Other risk scenarios in power grid operations should also be included.

[0064] Early warning information is transmitted through communication channels for feedback, such as SMS, email, APP notification, etc., to provide real-time feedback to relevant staff.

[0065] Example 2

[0066] On the basis of Example 1, this embodiment further provides an environment intelligent perception system based on pipeline coding and multi-branch decoding, including: The sensor data acquisition module is used to acquire multiple sensor monitoring image data.

[0067] The data processing module is used to process the image data monitored by each sensor to obtain the processed sensor monitoring image data. In some embodiments, the data processing module is specifically used to integrate, denoise and correct the image data monitored by each sensor to obtain the processed sensor monitoring image data.

[0068] The risk identification module is used to obtain visual identification results of various risks in the power grid working environment based on the processed sensor monitoring image data and the risk recognition model.

[0069] The risk perception model adopts an encoder-decoder structure.

[0070] The encoder is used to perform down-sampling operation on the processed sensor monitoring image data, which includes a basic convolution module and multiple feature encoding modules in sequence.

[0071] The decoder includes a main branch and multiple sub-branches. The output ends of the main branch and the sub-branches are connected to the detection module through the fusion module. The input end of the main branch is connected to the output end of the encoder, and multiple feature decoding modules are arranged on it. The input end of the feature decoding module is connected to the corresponding basic convolution module and the feature encoding module, and the output end is connected to the corresponding sub-branch. The sub-branches are all provided with feature decoding modules.

[0072] The basic convolution module and the feature encoding module are used to downsample the processed sensor monitoring image data, and the feature decoding module is used to upsample the high-level semantic information output by the encoder and the shallow detail information output by the basic convolution module and the feature encoding module.

[0073] The feature encoding module includes a P-Block encoder, a normalization layer, and an activation function layer in sequence.

[0074] Among them, the P-Block encoder includes a first convolutional layer, four intermediate convolutional layers and a second convolutional layer connected in sequence. The convolution kernel of the first convolutional layer is 1×1, which is used to expand the number of feature channels of the input image data. The four intermediate convolutional layers are 3×3 dilated convolutions with different dilation rates, which are used to reduce the output image data features to one-fourth of the input image features. The convolution kernel of the second convolutional layer is 1×1, which is used to perform feature fusion on the output image data of the spliced ​​first convolutional layer and the four intermediate convolutional layers. The activation function layer is the RELU activation function.

[0075] The operation expression of the P-Block encoder is as follows: ; in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

[0076] The number of sub-branches, feature encoding modules and feature decoding modules is 4, and the input end of the feature decoding module is connected to the ordinary convolution module and the first three feature encoding modules respectively.

[0077] The feature decoding module includes a fully connected layer, an upsampling module, a convolutional layer, and a batch normalization layer. The upsampling module is used to upsample the input image data using a bilinear interpolation method. The convolution kernel of the convolutional layer is , which is used to change the number of image data channels to 2.

[0078] The operation expression of the secondary branch is as follows: ; in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ; in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

[0079] An environmental intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module.

[0080] The environment intelligent perception enhancement module is used to enhance the environment detection part in the shallow detail information, which is divided into the first part and the second part.

[0081] The first part includes a maximum pooling layer and an average pooling layer in parallel, and the maximum pooling layer and the average convolution layer are both connected to the convolution layer. The input image feature data is fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and is dot-multiplied with the input image feature data to obtain the first part of output image feature data; The second part uses average pooling to collect long-term dependent context information in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts image features through the convolution layer and aggregates them through the fully connected layer, and then passes through the The convolution layer, the Sigmoid function layer and the first part of the output image feature data are multiplied to obtain the output image feature data.

[0082] The fusion module includes a fully connected layer and The convolution layer has the following operation expression: ; in, represents the output of the fusion module, express The convolution operation of size, The risk perception model of this embodiment further includes a convolutional attention module, which is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence; The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain the output image feature data. The operation flow of the channel attention module is expressed as follows: ; in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, represents the spatial attention weight, represents the output of the channel attention module; The operation flow of the spatial attention module is expressed as follows: ; in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

[0083] In some embodiments, it also includes an early warning module for generating early warning information for various risk scenarios according to the visual identification results of various risks in the power grid working environment, and transmitting it through a communication channel for feedback.

[0084] Embodiment 3: This embodiment provides a computer device, including: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of the environment intelligent perception method based on pipeline coding and multi-branch decoding described in Example 1.

[0085] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0086] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0089] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0090] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An environment intelligent perception method based on pipeline coding and multi-branch decoding, characterized in that: include: Acquire multiple sensor monitoring image data; Processing each sensor monitoring image data to obtain processed sensor monitoring image data; According to the processed sensor monitoring image data, the risk recognition model is used to obtain the visual recognition results of various risks in the power grid working environment.

2. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 1, characterized in that: The sensor includes at least one of a temperature sensor, a humidity sensor, a photosensor and a vibration sensor, wherein the temperature sensor is used to monitor the temperature of the working environment of the power grid, the humidity sensor is used to monitor the humidity of the working environment of the power grid, the photosensor is used to monitor the light intensity of the working environment of the power grid, and the vibration sensor is used to monitor the vibration of the power grid equipment.

3. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 1, characterized in that: Processing the monitoring image data of each sensor to obtain the processed sensor monitoring image data includes: integrating, denoising and correcting the monitoring image data of each sensor to obtain the processed sensor monitoring image data.

4. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 1, characterized in that: The risk perception model adopts an encoder-decoder structure; The encoder is used to perform a down-sampling operation on the processed sensor monitoring image data, which sequentially includes a basic convolution module and a plurality of feature encoding modules; The decoder includes a main branch and multiple sub-branches, the output ends of the main branch and the sub-branches are connected to the detection module through the fusion module, the input end of the main branch is connected to the output end of the encoder, and multiple feature decoding modules are arranged on it, the input end of the feature decoding module is connected to the corresponding basic convolution module and the feature encoding module, and the output end is connected to the corresponding sub-branch, and the sub-branch is provided with a feature decoding module; The basic convolution module and the feature encoding module are used to downsample the sensor monitoring image data after processing, and the feature decoding module is used to upsample the high-level semantic information output by the encoder and the shallow detail information output by the basic convolution module and the feature encoding module.

5. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 4 is characterized in that: The feature encoding module includes a P-Block encoder, a normalization layer, and an activation function layer in sequence; The P-Block encoder includes a first convolutional layer, four intermediate convolutional layers and a second convolutional layer connected in sequence, the convolution kernel of the first convolutional layer is 1×1, which is used to expand the number of feature channels of the input image data, the four intermediate convolutional layers are 3×3 dilated convolutions with different dilation rates, which are used to reduce the output image data features to one quarter of the input image features, the convolution kernel of the second convolutional layer is 1×1, which is used to perform feature fusion on the output image data of the spliced ​​first convolutional layer and the four intermediate convolutional layers, and the activation function layer is a RELU activation function; The operation expression of the P-Block encoder is as follows: ； in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

6. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 4, characterized in that: The number of the sub-branches, feature encoding modules and feature decoding modules is 4, and the input end of the feature decoding module is respectively connected to the common convolution module and the first three feature encoding modules; The feature decoding module includes a fully connected layer, an upsampling module, a convolution layer and a batch normalization layer in sequence. The upsampling module is used to upsample the input image data using a bilinear interpolation method. The convolution kernel of the convolution layer is , which is used to change the number of image data channels to 2; The operation expression of the sub-branch is as follows: ； in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ； in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

7. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 4, characterized in that: An environment intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module; The environment intelligent perception enhancement module is used to enhance the environment detection part in the shallow detail information, which is divided into a first part and a second part; The first part includes a maximum pooling layer and an average pooling layer in parallel, the maximum pooling layer and the average convolution layer are both connected to a convolution layer, the input image feature data are fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and are dot-multiplied with the input image feature data to obtain the first part of output image feature data; The second part uses average pooling to collect long-term dependent context information in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts image features through the convolution layer, aggregates them through the fully connected layer, and then passes through the The convolution layer, the Sigmoid function layer and the first part of the output image feature data are multiplied to obtain the output image feature data.

8. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 4, characterized in that: The fusion module includes a fully connected layer and The convolution layer has the following operation expression: ； in, represents the output of the fusion module, express The convolution operation of size, Represents a concatenation operation.

9. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 4, characterized in that: It also includes a convolutional attention module, which is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence; The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain output image feature data; The operation flow of the channel attention module is expressed as follows: ； in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, represents the spatial attention weight, represents the output of the channel attention module; The operation flow of the spatial attention module is expressed as follows: ； in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

10. An intelligent environmental perception system based on pipeline coding and multi-branch decoding, characterized in that: include: A sensor data acquisition module is used to acquire multiple sensor monitoring image data; A data processing module is used to process the monitoring image data of each sensor to obtain processed sensor monitoring image data; The risk identification module is used to obtain visual identification results of various risks in the power grid working environment based on the processed sensor monitoring image data and the risk recognition model.

11. The environment intelligent perception system based on pipeline coding and multi-branch decoding according to claim 10, characterized in that: The data processing module is specifically used to integrate, denoise and correct the image data monitored by each sensor to obtain processed sensor monitoring image data.

12. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 10, characterized in that: The risk perception model adopts an encoder-decoder structure; The encoder is used to perform a down-sampling operation on the processed sensor monitoring image data, which sequentially includes a basic convolution module and a plurality of feature encoding modules; The decoder includes a main branch and multiple sub-branches, the output ends of the main branch and the sub-branches are connected to the detection module through the fusion module, the input end of the main branch is connected to the output end of the encoder, and multiple feature decoding modules are arranged on it, the input end of the feature decoding module is connected to the corresponding basic convolution module and the feature encoding module, and the output end is connected to the corresponding sub-branch, and the sub-branch is provided with a feature decoding module; The basic convolution module and the feature encoding module are used to downsample the sensor monitoring image data after processing, and the feature decoding module is used to upsample the high-level semantic information output by the encoder and the shallow detail information output by the basic convolution module and the feature encoding module.

13. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 12, characterized in that: The feature encoding module includes a P-Block encoder, a normalization layer, and an activation function layer in sequence; The P-Block encoder includes a first convolutional layer, four intermediate convolutional layers and a second convolutional layer connected in sequence, the convolution kernel of the first convolutional layer is 1×1, which is used to expand the number of feature channels of the input image data, the four intermediate convolutional layers are 3×3 dilated convolutions with different dilation rates, which are used to reduce the output image data features to one quarter of the input image features, the convolution kernel of the second convolutional layer is 1×1, which is used to perform feature fusion on the output image data of the spliced ​​first convolutional layer and the four intermediate convolutional layers, and the activation function layer is a RELU activation function; The operation expression of the P-Block encoder is as follows: ； in, and Respectively and The convolution operation of size, represents the output image data of the first convolutional layer, Indicates The output image data of the intermediate convolutional layer, represents the output image data of the second convolutional layer, Represents a concatenation operation.

14. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 12, characterized in that: The number of the sub-branches, feature encoding modules and feature decoding modules is 4, and the input end of the feature decoding module is respectively connected to the common convolution module and the first three feature encoding modules; The feature decoding module includes a fully connected layer, an upsampling module, a convolution layer and a batch normalization layer in sequence. The upsampling module is used to upsample the input image data using a bilinear interpolation method. The convolution kernel of the convolution layer is 1×1, which is used to change the number of image data channels to 2. The operation expression of the sub-branch is as follows: ； in, Indicates The output of the feature decoding module on the secondary branch, represents the input of the main branch, Indicates The input of each branch, Express Upsampling is performed using bilinear interpolation; The operation expression of the main branch is as follows: ； in, Represents the output of the master branch, express The convolution operation of size, Represents a batch normalization operation.

15. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 12, characterized in that: An environment intelligent perception enhancement module is provided between the feature decoding module on the main branch and the corresponding basic convolution module and feature encoding module; The environment intelligent perception enhancement module is used to enhance the environment detection part in the shallow detail information, which is divided into a first part and a second part; The first part includes a maximum pooling layer and an average pooling layer in parallel, the maximum pooling layer and the average convolution layer are both connected to a convolution layer, the input image feature data are fused after passing through the maximum pooling layer, the average pooling layer and the convolution layer respectively, and are dot-multiplied with the input image feature data to obtain the first part of output image feature data; The second part uses average pooling to collect contextual information with long dependencies in the output image feature data of the first part in the horizontal and vertical dimensions, and extracts the image features through the convolution layer and then aggregates them through the fully connected layer, and then passes through the 1×1 convolution layer and the Sigmoid function layer in sequence to perform dot multiplication with the output image feature data of the first part to obtain the output image feature data.

16. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 12, characterized in that: The fusion module includes a fully connected layer and a 1×1 convolutional layer, and its operation expression is as follows: ； in, represents the output of the fusion module, express The convolution operation of size, Represents a concatenation operation.

17. The method for intelligent environmental perception based on pipeline coding and multi-branch decoding according to claim 12, characterized in that: It also includes a convolutional attention module, which is arranged between the output end of the encoder and the input end of the decoder, and includes a channel attention module and a spatial attention module connected in sequence; The channel attention module is used to generate an attention feature map on the channel dimension for the input image feature data, and then multiply it with the input image feature data and input it into the spatial attention module. The spatial attention module is used to generate an attention feature map on the spatial dimension, and then multiply it with the input image feature data to obtain output image feature data; The operation flow of the channel attention module is expressed as follows: ； in, represents the input of the channel attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. represents the Sigmoid activation function, represents the nonlinear fitting operation, represents the channel attention weight, represents the spatial attention weight, represents the output of the channel attention module; The operation flow of the spatial attention module is expressed as follows: ； in, represents the input of the spatial attention module, represents the output of the spatial attention module, Express Perform average pooling operation. Express Perform the maximum pooling operation. express The convolution operation of size .

18. A computer device, characterized in that: include: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of the environmental intelligent perception method based on pipeline coding and multi-branch decoding as described in any one of claims 1 to 9.

Images