Variable grain adaptive coding method for smoky dust disaster environment

Abstract

The application relates to the technical field of image coding, in particular to a variable-granularity adaptive coding method for a smoke dust disaster environment, which comprises the following steps: performing dust infrared transmittance dynamic compensation based on the current dust concentration and visibility of a smoke dust disaster environment scene obtained, and calculating a transmittance compensation coefficient; synchronously collecting a visible light image and a SWIR image of the smoke dust disaster environment scene, performing gain compensation on the SWIR image based on the transmittance compensation coefficient, and obtaining a compensated SWIR image; performing feature extraction and fusion on the visible light image and the compensated SWIR image, obtaining fused features, and restoring the fused features into a fused image; combining local texture complexity to select a DCT block size, performing DCT transformation on the fused image, and obtaining a group of DCT coefficients; applying a quantization parameter threshold constraint to abnormal high-frequency DCT coefficients, performing variable-granularity DCT quantization on the fused image, and obtaining entropy coding.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image coding, and particularly relate to a variable-granularity adaptive coding method for a smoke and dust disaster environment. BACKGROUND

[0002] In the production of coal mines, chemical plants and other industrial and mining enterprises, sudden disasters such as fire and explosion are often accompanied by a large amount of smoke and dust. For example, after the fire of the mine conveyor belt, the smoke produced by the combustion in the fire contains CO2, carbon particles and aerosols, and the fire spreads to the adjacent mining area, which will cause the ventilation system to reverse, and the CO concentration will spread to the whole mine within 2 hours, causing Mie scattering effect, and the smoke generated will cause the image in the visible light band to be blurred, causing the monitoring system to fail. The coal dust, rock dust and other solid particles in the mine disaster have a high absorption rate of up to 60% in the infrared band, which will cause the human body temperature signal to attenuate by more than 50%, and the infrared thermal imaging equipment cannot identify the heat source of the trapped personnel, and cannot accurately search and rescue. If the fire causes an explosion, the combined pollution of smoke and dust will interfere with multiple modal data sources such as visible light, infrared, and gas sensors. The high-temperature smoke flow in the fire will cause the infrared sensor to misjudge the position of the heat source, and the dust covering will make the visible light camera unable to capture the key structure collapse information, increasing the difficulty of rescue work.

[0003] At the same time, after the explosion in the mine disaster, the visibility in the mine and the plant area drops sharply, and the visibility will drop from 10 meters to 0.5 meters within 1 second, and the PM2.5 concentration will instantaneously exceed 1000 μg / m³. The CO concentration in the fire rises from 0.1% to 2% in only 30 seconds, and the oxygen concentration will drop from 21% to below 12%, which will cause the combustion state to change suddenly, and the gas concentration will fluctuate greatly. In addition, the temperature in the fire area can reach more than 800°C, but the temperature gradient in the smoke diffusion path presents a nonlinear distribution, with a temperature difference of 200°C per meter, and the parameter mutation and nonlinearity will make the changes of dynamic environmental parameters more complex.

[0004] The smoke and dust in the disaster will pose the following challenges to the safety production of enterprises.

[0005] First, causing the failure of enterprise safety production monitoring and early warning. In the smoke and dust environment, the false alarm rate of traditional cameras will increase by 40%, and dust clusters will be misjudged as flames, causing the fire warning to be delayed by more than 5 minutes. The sensitivity of gas sensors will decrease by 70% due to dust adhesion, and the methane detection error will reach ± 15%.

[0006] Second, increasing the safety risk of equipment and personnel. Dust accumulation will cause equipment short circuit explosion, and the failure rate of coal mine electrical equipment will increase by 3 times.

[0007] Third, production interruption and economic loss. A single dust over-standard accident can cause the production line to shut down for 8 to 24 hours, with direct losses exceeding one million yuan.

[0008] Fourth, making disaster rescue more complicated. When visibility is less than 1 meter due to the presence of smoke, the navigation positioning error of rescue robots can be as high as more than 3 meters. The effective detection distance of infrared life detection instruments in a dusty environment will be shortened to 30% of the original distance, and rescue path planning will be hindered. The temperature of high-temperature smoke flow is greater than 300℃, which can shorten the failure time of protective clothing to 10 minutes, and rescue personnel exposed to the outside for 15 minutes will appear blurred consciousness, and the risk of CO poisoning will increase sharply. When the dust concentration exceeds 300g / m³, static sparks can cause explosions, and the disorder of the ventilation system can increase the spread speed of fire by 50%.

[0009] In a smoke and dust disaster environment, it is very important to quickly and accurately identify key disaster characteristics. However, traditional image encoding methods are not adaptable to complex environments, and in low-visibility environments such as smoke and dust, they cannot effectively identify key disaster characteristics (such as infrared heat sources and human silhouettes), resulting in distorted data transmission. The encoding mode of traditional image encoding methods is fixed and cannot be dynamically optimized in combination with real-time meteorological parameters (visibility, particulate matter concentration), resulting in insufficient penetration, which will cause bandwidth waste and loss of key information. SUMMARY

[0010] To solve the above technical problems, the embodiments of the present application propose a variable-granularity adaptive encoding method for smoke and dust disaster environments, which aims to use dust infrared transmittance dynamic compensation technology and multispectral fusion encoding technology to improve heat source identification rate, significantly reduce the bandwidth required in the encoding process, improve the encoding speed, and dynamically optimize and enhance in combination with real-time meteorological parameters, which well meets the high real-time and high-precision emergency response requirements in smoke and dust disaster environment scenarios.

[0011] To achieve the above object, the embodiment of the present application proposes a variable-granularity adaptive coding method for a smog dust disaster environment, which comprises the following steps: acquiring the current dust concentration and visibility of the smog dust disaster environment scene, performing dust infrared transmittance dynamic compensation based on the dust concentration and the visibility when the dust concentration is greater than a preset first dust concentration threshold or the visibility is less than a preset first visibility threshold, and calculating a transmittance compensation coefficient; synchronously starting a visible light camera and a SWIR camera to shoot the smog dust disaster environment scene, obtaining a visible light image and a SWIR image of the smog dust disaster environment scene, and performing gain compensation on the SWIR image based on the transmittance compensation coefficient to obtain a compensated SWIR image; performing feature extraction on the visible light image and the compensated SWIR image respectively to obtain visible light features and SWIR features, performing feature fusion on the visible light features and the SWIR features based on an attention mechanism and a waveband weight matrix to obtain fusion features, and restoring the fusion features to a fusion image; combining a local texture complexity based on a Zernike matrix to select a DCT block size, performing DCT transformation on the fusion image based on the selected DCT block size to obtain a group of DCT coefficients; detecting abnormal high-frequency coefficients based on a mapping relationship between dust particle sizes and the DCT coefficients, imposing a quantization parameter threshold constraint on the abnormal high-frequency coefficients, and performing variable-granularity DCT quantization on the fusion image based on a quantization parameter and the quantization parameter threshold constraint to obtain an entropy coding of the smog dust disaster environment scene.

[0012] To achieve the above object, the embodiment of the present application also proposes a variable granularity adaptive coding system for a smog dust disaster environment, which comprises a dust concentration sensor, an visibility observation instrument, a transmittance compensation coefficient calculation module, an image acquisition and compensation module, a visible light camera, a SWIR camera, a feature extraction and feature fusion module, a DCT transformation module, a dust suppression module of variable granularity coding, and a DCT quantization module; the dust concentration sensor is used to acquire the current dust concentration and the current dust particle size of the smog dust disaster environment scene; the visibility observation instrument is used to acquire the current visibility of the smog dust disaster environment scene; the transmittance compensation coefficient calculation module is used to detect whether the dust concentration is greater than a preset first dust concentration threshold and whether the visibility is less than a preset first visibility threshold, and in the case that the dust concentration is greater than the preset first dust concentration threshold or the visibility is less than the preset first visibility threshold, the dust infrared transmittance is dynamically compensated based on the dust concentration and the visibility, and the transmittance compensation coefficient is calculated; the image acquisition and compensation module is used to synchronously start the visible light camera and the SWIR camera to shoot the smog dust disaster environment scene, to obtain the visible light image and the SWIR image of the smog dust disaster environment scene, and to perform gain compensation on the SWIR image based on the transmittance compensation coefficient, to obtain the compensated SWIR image; the feature extraction and feature fusion module is used to respectively extract features from the visible light image and the compensated SWIR image, to obtain visible light features and SWIR features, to perform feature fusion on the visible light features and the SWIR features based on an attention mechanism and a waveband weight matrix, to obtain fusion features, and to restore the fusion features into a fusion image; the DCT transformation module is used to select a DCT block size based on a local texture complexity based on a Zernike matrix, to perform DCT transformation on the fusion image based on the selected DCT block size, to obtain a group of DCT coefficients; the dust suppression module of variable granularity coding is used to detect abnormal high-frequency coefficients based on a mapping relationship between the dust particle size and the DCT coefficients, and to apply a quantization parameter threshold constraint to the abnormal high-frequency coefficients; and the DCT quantization module is used to perform DCT quantization of variable granularity on the fusion image based on the quantization parameter and the quantization parameter threshold constraint, to obtain an entropy coding of the smog dust disaster environment scene.

[0013] To achieve the above object, the embodiment of the present application also provides an electronic device, which comprises at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions capable of being executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned variable granularity adaptive coding method for a smog dust disaster environment.

[0014] To achieve the above object, the embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program can realize the variable-granularity adaptive coding method for a smoke dust disaster environment when executed by a processor.

[0015] The variable-granularity adaptive coding method for a smoke dust disaster environment provided by the present application makes targeted improvements on the traditional image coding method from three aspects of dust infrared transmittance dynamic compensation, multi-spectrum fusion coding and dust suppression of variable granularity. The traditional dust concentration compensation technology can only adjust the global brightness, while the present application performs dust infrared transmittance dynamic compensation based on the dust concentration and the visibility, so that the heat source features can be recognized in the smoke dust, the signal-to-noise ratio of the heat source after compensation is greatly improved, and the detection and discovery efficiency of the human target is improved. In the aspect of multi-spectrum utilization, the present application performs gain compensation on the SWIR image, performs feature extraction, fusion and restoration based on the visible light image and the compensated SWIR image, obtains a fused image which retains the waveband with strong penetration, and uses the fused image as the basic data for subsequent coding, thereby well enhancing the adaptability to the smoke dust disaster environment. In the aspect of suppression mechanism, the traditional image coding method usually adopts the most basic fixed low-pass filtering technology, which lacks flexibility, while the present application performs detection on the abnormal high-frequency coefficients based on the mapping relationship between the dust particle size and the DCT coefficients, and applies a quantization parameter threshold constraint on the abnormal high-frequency coefficients, so as to realize adaptive filtering and greatly enhance the penetration ability to the smoke dust. In summary, the present application effectively improves the heat source recognition rate, greatly reduces the bandwidth required in the image coding process, improves the coding speed, and can realize dynamic optimization and enhancement combined with real-time meteorological parameters, thereby well meeting the high real-time and high-precision emergency response requirements in the smoke dust disaster environment.

[0016] Optionally, the dust concentration and the visibility of the smoke dust disaster environment scene are acquired, and when the dust concentration is greater than a preset first dust concentration threshold or the visibility is less than a preset first visibility threshold, dust infrared transmittance dynamic compensation is performed based on the dust concentration and the visibility, including: acquiring the dust concentration and the visibility of the smoke dust disaster environment scene, detecting whether the dust concentration is greater than a preset first dust concentration threshold and detecting whether the visibility is less than a preset first visibility threshold; if it is detected that the dust concentration is greater than the preset first dust concentration threshold or it is detected that the visibility is less than the preset first visibility threshold, performing dust infrared transmittance dynamic compensation based on the dust concentration and the visibility; and if it is detected that the dust concentration is not greater than the preset first dust concentration threshold and it is detected that the visibility is not less than the preset first visibility threshold, performing normal coding.

[0017] Optionally, the dust infrared transmittance dynamic compensation is performed based on the dust concentration and the visibility, and a transmittance compensation coefficient is calculated, including:

[0018] A code rate adjustment coefficient is determined based on the dust concentration and the visibility by the following formula:

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] wherein, and respectively represent the current dust concentration and the visibility, , and respectively represent the preset first dust concentration threshold, the second dust concentration threshold and the third dust concentration threshold, , and respectively represent the preset first visibility threshold, the second visibility threshold and the third visibility threshold, represents the determined code rate adjustment coefficient, , and respectively represent the preset first code rate adjustment coefficient, the second code rate adjustment coefficient and the third code rate adjustment coefficient;

[0024] The transmittance compensation coefficient is calculated based on the code rate adjustment coefficient, the preset channel number and the depth of field information of the smoke dust disaster environment scene by the following formula:

[0025] ;

[0026] wherein, represents the preset channel number, represents the depth of field information of the smoke dust disaster environment scene, represents the calculated transmittance compensation coefficient.

[0027] Optionally, the SWIR image is gain compensated based on the transmittance compensation coefficient by the following formula to obtain a compensated SWIR image:

[0028] ;

[0029] wherein, represents the SWIR image, represents the compensated SWIR image.

[0030] Optionally, feature extraction is performed on the visible light image and the compensated SWIR image respectively to obtain visible light features and SWIR features, and feature fusion is performed on the visible light features and the SWIR features based on an attention mechanism and a band weight matrix to obtain fused features, including:

[0031] The visible light image and the compensated SWIR image are input into a pre-trained double-branch feature extraction model, feature extraction is performed on by a visible light branch of the double-branch feature extraction model to obtain visible light features , and feature extraction is performed on by a thermal infrared branch of the double-branch feature extraction model to obtain thermal infrared features ; wherein the network structure of the visible light branch is ResNet-18, and the network structure of the thermal infrared branch is an improved U-Net.

[0032] The thermal infrared features are reorganized by weighting based on the band weight matrix to obtain band weight weighted thermal infrared features :

[0033] ;

[0034] ;

[0035] ;

[0036] wherein represents the band corresponding to the point of the visible light features , and represents the band weight corresponding to the point of the thermal infrared features .

[0037] The visible light features and the thermal infrared features are fused based on an attention mechanism by taking as a query vector, taking as a key vector and a value vector to obtain fused features:

[0038] ;

[0039] ;

[0040] ;

[0041] wherein, 、 、 respectively represent a query vector, a key vector and a value vector, represents a dimension, and the upper right corner represents a transpose operation, represents a Softmax function, represents a fusion feature.

[0042] Optionally, the DCT block size is selected in combination with the local texture complexity based on the Zernike moment, and the method comprises the following steps:

[0043] The Zernike moment is calculated based on the visible light image by the following formula:

[0044] ;

[0045] wherein, represents a visible light image with a size of , represents a pixel value of a point in , represents a Zernike polynomial of order with a repetition rate of corresponding to the point , represents a calculated Zernike moment of order with a repetition rate of ;

[0046] The texture complexity is calculated based on by the following formula:

[0047] ;

[0048] wherein, represents a total order number, represents a calculated texture complexity;

[0049] The DCT block size is selected based on by the following formula:

[0050] ;

[0051] ;

[0052] wherein, and are respectively a preset first texture complexity threshold and a second texture complexity threshold, represents the selected DCT block size.

[0053] Optionally, the abnormal high frequency coefficient is detected based on the mapping relationship between the dust particle size and the DCT coefficient, and a quantization parameter threshold constraint is applied to the abnormal high frequency coefficient, including: determining an abnormal high frequency threshold based on the mapping relationship between the dust particle size and the DCT coefficient; comparing each DCT coefficient with the abnormal high frequency threshold, determining the DCT coefficient smaller than the abnormal high frequency threshold as a normal coefficient, and determining the DCT coefficient greater than or equal to the abnormal high frequency threshold as an abnormal high frequency coefficient; for the normal coefficient, a preset conventional quantization parameter is allocated; for the abnormal high frequency coefficient, a quantization parameter threshold constraint is applied on the basis of the conventional quantization parameter, to obtain a constrained quantization parameter and allocate it to the abnormal high frequency coefficient. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the following drawings are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings. The drawings described herein are only used to explain the present application, and are not used to limit the present application.

[0055] Figure 1 is a flow chart of a variable granularity adaptive coding method for a smoke dust disaster environment provided in an embodiment of the present application;

[0056] Figure 2 is a flow chart of selecting conventional coding or infrared feature enhanced coding based on dust concentration and visibility provided in an embodiment of the present application;

[0057] Figure 3 is a schematic diagram of feature extraction, fusion and restoration process provided in an embodiment of the present application;

[0058] Figure 4 is an entropy coding thermodynamic diagram provided in an embodiment of the present application;

[0059] Figure 5 is a structural schematic diagram of a variable granularity adaptive coding system for a smoke dust disaster environment provided in another embodiment of the present application;

[0060] Figure 6 is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. Those skilled in the art can understand that, in the embodiments of the present application, many technical details are proposed in order to make the readers better understand. However, the technical solutions claimed by the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation modes of the present application. The following embodiments can be combined and referenced with each other without contradiction.

[0062] One embodiment of the present application proposes a variable-granularity adaptive coding method for a smoke dust disaster environment, which is applied to an electronic device. The electronic device can be a terminal or a server. In the embodiments of the present application and the following embodiments, the server is taken as an example for description. The implementation details of the variable-granularity adaptive coding method for a smoke dust disaster environment proposed in the present embodiment will be described in detail below. The following content only provides implementation details for the convenience of understanding, and is not necessary for implementing the present solution.

[0063] The specific process of the variable-granularity adaptive coding method for a smoke dust disaster environment proposed in the present embodiment can be as shown in Figure 1 The specific process of the variable-granularity adaptive coding method for a smoke dust disaster environment proposed in the present embodiment can be as shown in

[0064] In step 11, the current dust concentration and visibility of the smoke dust disaster environment scene are obtained. When the dust concentration is greater than a preset first dust concentration threshold or the visibility is less than a preset first visibility threshold, dust infrared transmittance dynamic compensation is performed based on the dust concentration and the visibility, and a transmittance compensation coefficient is calculated.

[0065] In a specific implementation, in the face of a smoke dust disaster environment scene, the server first needs to obtain the current dust concentration through a dust concentration sensor, and obtain the current visibility through a visibility observation instrument. Then, threshold comparison is performed on the current dust concentration and the current visibility respectively. When the current dust concentration is greater than a preset first dust concentration threshold, or the current visibility is less than a preset first visibility threshold, dust infrared transmittance dynamic compensation is performed based on the dust concentration and the visibility, and a transmittance compensation coefficient is calculated.

[0066] In one example, after obtaining the current dust concentration and the current visibility of the smog and dust disaster environment scene, the server detects whether the current dust concentration is greater than a preset first dust concentration threshold and whether the current visibility is less than a preset first visibility threshold. If it is detected that the current dust concentration is greater than the preset first dust concentration threshold or that the current visibility is less than the preset first visibility threshold, dust infrared transmittance dynamic compensation is performed based on the current dust concentration and the current visibility, and a transmittance compensation coefficient is calculated, that is, infrared feature enhancement coding is selected. If it is detected that the current dust concentration is not greater than the preset first dust concentration threshold and that the current visibility is not less than the preset first visibility threshold, dust infrared transmittance dynamic compensation is not needed, and regular coding can be directly performed, which effectively saves the computing resources and improves the coding efficiency to a certain extent.

[0067] In one example, the process of selecting regular coding or infrared feature enhancement coding based on the dust concentration and the visibility is as shown in Figure 2 In one example, the process of selecting regular coding or infrared feature enhancement coding based on the dust concentration and the visibility is as shown in

[0068] In one example, in the process of performing dust infrared transmittance dynamic compensation based on the dust concentration and the visibility to calculate the transmittance compensation coefficient, the code rate adjustment coefficient is first determined based on the dust concentration and the visibility by the following formula:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] wherein, and represent the current dust concentration and the current visibility, , and represent the preset first dust concentration threshold, the second dust concentration threshold and the third dust concentration threshold, , and ​respectively represent a preset first visibility threshold, a second visibility threshold and a third visibility threshold, represents the determined code rate adjustment coefficient, 、 and respectively represent a preset first code rate adjustment coefficient, a second code rate adjustment coefficient and a third code rate adjustment coefficient.

[0074] In one example, is 500 μg / m³, is 750 μg / m³, is 1000 μg / m³.

[0075] In one example, is 5 m, is 3 m, is 1 m.

[0076] In one example, is 1.2, is 1.6, is 2.0.

[0077] In one example, after the server calculates the transmittance compensation coefficient, the server can calculate the transmittance compensation coefficient based on the calculated code rate adjustment coefficient, the preset number of channels and the depth of field information of the smoke and dust disaster environment scene:

[0078] ;

[0079] wherein, represents the preset number of channels, represents the depth of field information of the smoke and dust disaster environment scene, represents the calculated transmittance compensation coefficient.

[0080] Step 12, synchronously start the visible light camera and the SWIR camera to capture the smoke and dust disaster environment scene to obtain a visible light image and a SWIR image of the smoke and dust disaster environment scene, and perform gain compensation on the SWIR image based on the transmittance compensation coefficient to obtain a compensated SWIR image.

[0081] In a specific implementation, after the server determines that infrared feature enhancement coding is needed, the server can synchronously start the visible light camera and the SWIR camera to capture the smoke and dust disaster environment scene to obtain a visible light image and a SWIR image of the smoke and dust disaster environment scene, and perform gain compensation on the SWIR image based on the transmittance compensation coefficient to obtain a compensated SWIR image. The heat source in the compensated SWIR image is identifiable, and the heat source signal-to-noise ratio can be improved by 22 dB.

[0082] In one example, the visible light camera can be SONY IMX585, and the SWIR camera can be InGaAs sensor. The synchronization mode of the two cameras is hardware synchronization, and the time deviation is less than 1ms.

[0083] In one example, the server can perform gain compensation on the SWIR image based on the transmittance compensation coefficient by the following formula to obtain the compensated SWIR image:

[0084] ;

[0085] wherein, represents the SWIR image, represents the compensated SWIR image.

[0086] Step 13, respectively, feature extraction is performed on the visible light image and the compensated SWIR image to obtain visible light features and SWIR features. Feature fusion based on attention mechanism and band weight matrix is performed on the visible light features and the SWIR features to obtain fusion features, and the fusion features are restored to a fusion image.

[0087] In a specific implementation, after obtaining the compensated SWIR image, the server can perform feature extraction on the visible light image and the compensated SWIR image respectively to obtain visible light features and SWIR features. Then, feature fusion based on attention mechanism and band weight matrix is performed on the visible light features and the SWIR features to obtain fusion features. Finally, the fusion features are restored to a fusion image.

[0088] In one example, the feature extraction, fusion, and restoration process can be as shown in Figure 3 . The feature extraction is realized by a double-branch feature extraction model, which is composed of a visible light branch and a thermal infrared branch. The server inputs the visible light image and the compensated SWIR image into the pre-trained double-branch feature extraction model. The visible light branch of the double-branch feature extraction model is used to perform feature extraction on to obtain visible light features . The thermal infrared branch of the double-branch feature extraction model is used to perform feature extraction on to obtain thermal infrared features . The network structure of the visible light branch is ResNet-18, and the network structure of the thermal infrared branch is an improved U-Net.

[0089] In one example, the feature fusion process is realized by a fusion model, which is composed of a weighted recombination unit and an attention unit. The weighted recombination unit performs weighted recombination on the thermal infrared features based on the band weight matrix The band weight weighted thermal infrared features are obtained through band weight reorganization The attention unit is responsible for feature fusion based on the attention mechanism as the query vector, and as the key vector and the value vector, and and are fused to obtain the fused features .

[0090] In one example, the weighted reorganization unit reorganizes based on the band weight matrix to obtain the band weight weighted thermal infrared features :

[0091] ;

[0092] ;

[0093] ;

[0094] wherein represents the band corresponding to the point of , and represents the band weight corresponding to the point of .

[0095] In one example, the attention unit takes as the query vector, takes as the key vector and the value vector, and fuses and based on the attention mechanism to obtain the fused features :

[0096] ;

[0097] ;

[0098] ;

[0099] wherein , , respectively represent the query vector, the key vector, and the value vector, represents the dimension, and the upper right corner mark represents the transpose operation, represents the Softmax function.

[0100] In one example, the feature reduction process is implemented based on a reduction model, which in turn is based on fused features. Feature restoration is performed to obtain the fused image. .

[0101] Step 14: Select the DCT block size based on the local texture complexity based on Zernike moments, and perform DCT transformation on the fused image based on the selected DCT block size to obtain a set of DCT coefficients.

[0102] In the specific implementation, the server obtains the fused image. Then, the DCT block size can be selected based on the local texture complexity of the Zernike moments, and the fused image can then be processed based on the selected DCT block size. Perform DCT transformation to obtain a set of DCT coefficients.

[0103] In one example, the server calculates the Zernike moment based on a visible light image using the following formula:

[0104] ;

[0105] in, Indicates the size is Visible light images, express One point pixel values, Point The corresponding repetition rate of Zernike polynomial of order 1 This indicates the calculated repetition rate. of Zernike moment of order.

[0106] In the calculation Then, based on the following formula, Calculate the texture complexity:

[0107] ;

[0108] in, Indicates the total order. This represents the calculated texture complexity.

[0109] Finally, based on the following formula, Select the DCT block size:

[0110] ;

[0111] ;

[0112] wherein, and are preset first and second texture complexity thresholds, respectively, denotes a selected DCT block size.

[0113] At step 15, the abnormal high-frequency coefficients are detected based on the mapping relationship between the dust particle size and the DCT coefficients, a quantization parameter threshold constraint is imposed on the abnormal high-frequency coefficients, and variable-granularity DCT quantization is performed on the fused image based on the quantization parameter and the quantization parameter threshold constraint to obtain the entropy coding of the smoke and dust disaster environment scene.

[0114] In a specific implementation, after the DCT transformation is completed, the DCT quantization can be prepared. The server first detects the abnormal high-frequency coefficients based on the mapping relationship between the dust particle size and the DCT coefficients, imposes a quantization parameter threshold constraint on the abnormal high-frequency coefficients, and then performs variable-granularity DCT quantization on the fused image based on the quantization parameter and the quantization parameter threshold constraint, thereby obtaining the entropy coding of the smoke and dust disaster environment scene.

[0115] In one example, the entropy coding heat map of the smoke and dust disaster environment scene is as shown in Figure 4 .

[0116] In one example, for the DCT coefficients obtained by the DCT transformation, the server first determines the abnormal high-frequency threshold based on the preset mapping relationship between the dust particle size and the DCT coefficients. Then, the server compares each DCT coefficient with the abnormal high-frequency threshold in sequence, determines the DCT coefficients smaller than the abnormal high-frequency threshold as normal coefficients, and determines the DCT coefficients greater than or equal to the abnormal high-frequency threshold as abnormal high-frequency coefficients. For the normal coefficients, the server assigns a preset regular quantization parameter. For the abnormal high-frequency coefficients, the server needs to impose a quantization parameter threshold constraint on the basis of the regular quantization parameter, obtain the constrained quantization parameter, and assign it to the abnormal high-frequency coefficients.

[0117] In one example, the dust particle size is detected by a dust concentration sensor, and the mapping relationship between the preset dust particle size and the DCT coefficients records the minimum and maximum values of the DCT coefficients corresponding to the dust particle size. The server takes the maximum value of the DCT coefficients corresponding to the dust particle size as the abnormal high-frequency threshold.

[0118] The embodiment proposes a variable granularity adaptive coding method for a smoke dust disaster environment. The method is improved in three aspects, i.e., dust infrared transmittance dynamic compensation, multi-spectral fusion coding and dust suppression of variable granularity coding. The traditional dust concentration compensation technology can only adjust the global brightness. The embodiment is based on dust concentration and visibility to perform dust infrared transmittance dynamic compensation, so that the heat source feature can be identified in the smoke dust. The signal-to-noise ratio of the compensated heat source is greatly improved, and the detection and discovery efficiency of the human target is improved. In the aspect of multi-spectral utilization, the SWIR image is gain compensated, and the visible light image and the compensated SWIR image are used for feature extraction, feature fusion and restoration to obtain a fusion image with strong penetration, which is used as the basis data for subsequent coding, and the adaptability to the smoke dust disaster environment is improved. The traditional image coding method usually uses the most basic fixed low-pass filtering technology, and the flexibility is insufficient. The embodiment is based on the mapping relationship between the dust particle size and the DCT coefficient to detect the abnormal high-frequency coefficient, and the quantization parameter threshold constraint is applied to the abnormal high-frequency coefficient to realize adaptive filtering, and the penetration ability to the smoke dust is greatly improved. In summary, the variable granularity adaptive coding method for the smoke dust disaster environment improves the heat source recognition rate, greatly reduces the bandwidth required in the image coding process, improves the coding speed, and can be dynamically optimized and enhanced combined with real-time meteorological parameters, and meets the high real-time and high-precision emergency response requirements in the smoke dust disaster environment.

[0119] The steps of the above methods are only for clear description, and can be combined into one step or some steps can be divided into multiple steps in implementation, as long as the same logical relationship is included, which is within the protection scope of the application. Adding insignificant modifications or introducing insignificant designs in the algorithm or process, but not changing the core design of the algorithm and process, are within the protection scope of the application.

[0120] Another embodiment of the application proposes a variable granularity adaptive coding system for a smoke dust disaster environment. The details of the variable granularity adaptive coding system for the smoke dust disaster environment are described below. The following content is only the implementation details provided for easy understanding, and is not necessary for implementing the embodiment. Figure 5is a structural schematic diagram of a variable granularity adaptive coding system for a smoke dust disaster environment proposed in the embodiment, comprising: a dust concentration sensor 21, an visibility observation instrument 22, a transmittance compensation coefficient calculation module 31, an image acquisition and compensation module 32, a visible light camera 23, a SWIR camera 24, a feature extraction and feature fusion module 33, a DCT transformation module 34, a dust suppression module 35 of variable granularity coding and a DCT quantization module 36.

[0121] The dust concentration sensor 21 is used to obtain the current dust concentration and the current dust particle size of the smoke dust disaster environment scene.

[0122] The visibility observation instrument 22 is used to obtain the current visibility of the smoke dust disaster environment scene.

[0123] The transmittance compensation coefficient calculation module 31 is used to detect whether the dust concentration is greater than a preset first dust concentration threshold and detect whether the visibility is less than a preset first visibility threshold, and in the case that the dust concentration is greater than the preset first dust concentration threshold or the visibility is less than the preset first visibility threshold, the dust infrared transmittance is dynamically compensated based on the dust concentration and the visibility, and the transmittance compensation coefficient is calculated.

[0124] The image acquisition and compensation module 32 is used to synchronously start the visible light camera 23 and the SWIR camera 24 to shoot the smoke dust disaster environment scene, obtain the visible light image and the SWIR image of the smoke dust disaster environment scene, and perform gain compensation on the SWIR image based on the transmittance compensation coefficient generated by the transmittance compensation coefficient calculation module 31, to obtain the compensated SWIR image.

[0125] The feature extraction and feature fusion module 33 is used to respectively extract features from the visible light image and the compensated SWIR image to obtain visible light features and SWIR features, perform feature fusion on the visible light features and the SWIR features based on an attention mechanism and a waveband weight matrix to obtain fusion features, and restore the fusion features to a fusion image.

[0126] The DCT transformation module 34 is used to select a DCT block size in combination with a local texture complexity based on a Zernike matrix, perform DCT transformation on the fusion image based on the selected DCT block size, and obtain a group of DCT coefficients.

[0127] The dust suppression module 35 of variable granularity coding is used to detect abnormal high frequency coefficients based on the mapping relationship between the dust particle size and the DCT coefficients, and apply a quantization parameter threshold constraint to the abnormal high frequency coefficients.

[0128] The DCT quantization module 36 is configured to perform variable-granularity DCT quantization on the fused image based on a quantization parameter and a quantization parameter threshold constraint to obtain entropy coding of the smoke and dust disaster environment scene.

[0129] It can be found that the embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and the embodiment can be implemented in cooperation with the above-mentioned method embodiment. The related technical details and technical effects mentioned in the above-mentioned method embodiment are still valid in the embodiment. In order to reduce repetition, they will not be described here. Accordingly, the related technical details mentioned in the embodiment can also be applied to the above-mentioned method embodiment.

[0130] It is worth mentioning that each module and module involved in the embodiment is a logical module. In actual application, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units not closely related to solving the technical problems proposed in the present application are not introduced in the embodiment, but this does not mean that there are no other units in the embodiment.

[0131] Another embodiment of the present application provides an electronic device, as shown in the figure, comprising: at least one processor 41; and a memory 42 connected in communication with the at least one processor 41; wherein the memory 42 stores instructions executable by the at least one processor 41, and the instructions are executed by the at least one processor 41 to enable the at least one processor 41 to perform a variable-granularity adaptive coding method for a smoke and dust disaster environment as described in the above-mentioned method embodiment. Figure 6

[0132] Wherein, the memory and the processor are connected in a bus mode, the bus includes any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers, and power management circuits together, which are well known in the art, and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0133] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. And the memory can be used to store the data used by the processor in the execution operation. ​

[0134] Another embodiment of the present application provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program, when executed by a processor, enables a variable-granularity adaptive coding method for a smog dust disaster environment to be implemented.

[0135] That is, a person skilled in the art can understand that all or part of the steps in the above method embodiments can be completed by programs instructing relevant hardware, and the programs are stored in a storage medium and include a plurality of instructions for enabling a device (such as a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the method embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0136] A person skilled in the art can understand that the above embodiments are specific embodiments of the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application. For those skilled in the art, a number of improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements are also considered to be within the protection scope of the present application.

Claims

1. A variable-granularity adaptive coding method for smoke and dust disaster environments, characterized in that, The method includes: The current dust concentration and visibility in the smoke and dust disaster environment are obtained. When the dust concentration is greater than the preset first dust concentration threshold or the visibility is less than the preset first visibility threshold, dynamic compensation of dust infrared transmittance is performed based on dust concentration and visibility, and the transmittance compensation coefficient is calculated. The visible light camera and SWIR camera are simultaneously activated to capture images of the smoke and dust disaster environment scene, obtaining visible light images and SWIR images of the smoke and dust disaster environment scene. The SWIR image is then gain-compensated based on the transmittance compensation coefficient to obtain the compensated SWIR image. Feature extraction is performed on the visible light image and the compensated SWIR image respectively to obtain visible light features and SWIR features. Feature fusion based on attention mechanism and band weight matrix is ​​performed on the visible light features and SWIR features to obtain fused features, and then the fused image is restored. The DCT block size is selected by combining the local texture complexity based on Zernike moments, and the fused image is subjected to DCT transformation based on the selected DCT block size to obtain a set of DCT coefficients; Abnormal high-frequency coefficients are detected based on the mapping relationship between dust particle size and DCT coefficients. A quantization parameter threshold constraint is applied to the abnormal high-frequency coefficients. Based on the quantization parameter and the quantization parameter threshold constraint, the fused image is subjected to variable-granularity DCT quantization to obtain the entropy coding of the smoke and dust disaster environment scene. Abnormal high-frequency coefficients are detected based on the mapping relationship between dust particle size and DCT coefficients. A threshold constraint of quantization parameters is applied to the abnormal high-frequency coefficients, including: Based on the mapping relationship between dust particle size and DCT coefficient, the threshold for abnormal high frequency is determined; Each DCT coefficient is compared with an abnormal high-frequency threshold. DCT coefficients that are less than the abnormal high-frequency threshold are identified as normal coefficients, and DCT coefficients that are greater than or equal to the abnormal high-frequency threshold are identified as abnormal high-frequency coefficients. For normal coefficients, assign preset conventional quantization parameters; For abnormally high-frequency coefficients, a threshold constraint on the quantization parameters is applied on the basis of the regular quantization parameters to obtain the constrained quantization parameters, which are then assigned to the abnormally high-frequency coefficients.

2. The variable-granularity adaptive coding method for smoke and dust disaster environments according to claim 1, characterized in that, The system acquires the current dust concentration and visibility in a smoke and dust disaster environment. When the dust concentration exceeds a preset first dust concentration threshold or the visibility falls below a preset first visibility threshold, dynamic compensation for dust infrared transmittance is performed based on the dust concentration and visibility, including: The system acquires the current dust concentration and visibility in the smoke and dust disaster environment, detects whether the dust concentration is greater than a preset first dust concentration threshold, and detects whether the visibility is less than a preset first visibility threshold. If the detected dust concentration is greater than the preset first dust concentration threshold, or the detected visibility is less than the preset first visibility threshold, then dynamic compensation of dust infrared transmittance is performed based on dust concentration and visibility. If the detected dust concentration is not greater than the preset first dust concentration threshold, and the detected visibility is not less than the preset first visibility threshold, then regular coding is performed.

3. The variable granularity adaptive coding method for smoke and dust disaster environments according to claim 2, characterized in that, Dynamic compensation for dust infrared transmittance is performed based on dust concentration and visibility, and the transmittance compensation coefficient is calculated, including: The bitrate adjustment factor is determined using the following formula, based on dust concentration and visibility: ； ； ； ； in, and These represent the current dust concentration and visibility, respectively. , and These represent the preset first dust concentration threshold, second dust concentration threshold, and third dust concentration threshold, respectively. , and These represent the preset first visibility threshold, second visibility threshold, and third visibility threshold, respectively. This represents the determined bitrate adjustment factor. , and These represent the preset first bitrate adjustment coefficient, second bitrate adjustment coefficient, and third bitrate adjustment coefficient, respectively. The transmittance compensation coefficient is calculated using the following formula, based on the bitrate adjustment factor, the preset number of channels, and the depth information of the smoke and dust disaster environment: ； in, This indicates the preset number of channels. This represents depth information in environmental scenes depicting smoke and dust hazards. This represents the calculated transmittance compensation coefficient.

4. The variable-granularity adaptive coding method for smoke and dust disaster environments according to claim 3, characterized in that, The following formula is used to perform gain compensation on the SWIR image based on the transmittance compensation coefficient, resulting in the compensated SWIR image: ； in, Represents a SWIR image. This represents the compensated SWIR image.

5. The variable-granularity adaptive coding method for smoke and dust disaster environments according to claim 4, characterized in that, Feature extraction is performed on the visible light image and the compensated SWIR image separately to obtain visible light features and SWIR features. These features are then fused using an attention mechanism and a band weight matrix to obtain fused features, including: Visible light images and the compensated SWIR image The input is fed into a pre-trained two-branch feature extraction model, and the visible light branch pairs of the two-branch feature extraction model are used for processing. Feature extraction is performed to obtain visible light features. The thermal infrared branch of the dual-branch feature extraction model Feature extraction was performed to obtain thermal infrared features. The visible light branch uses ResNet-18 as its network structure, while the thermal infrared branch uses an improved U-Net. Based on the band weight matrix, the following formula is used. right Weighted recombination is performed to obtain the band-weighted thermal infrared features. : ； ； ； in, express point The corresponding band, express point Corresponding band weights; Based on the attention mechanism, the following formula is used to... As a query vector, As key vectors and value vectors, for and Perform feature fusion to obtain fused features: ； ； ； in, , , These represent the query vector, key vector, and value vector, respectively. Indicates dimension, upper right subscript This indicates the transpose operation. This represents the Softmax function. This indicates the fusion feature.

6. A variable-granularity adaptive coding method for smoke and dust disaster environments according to any one of claims 1 to 5, characterized in that, The DCT block size is selected based on Zernike moment-based local texture complexity, including: The Zernike moment is calculated based on a visible light image using the following formula: ； in, Indicates the size is Visible light images, express One point pixel values, Point The corresponding repetition rate of Zernike polynomial of order 1 This indicates the calculated repetition rate. of Zernike moments of order; Based on the following formula, Calculate the texture complexity: ； in, Indicates the total order. This represents the calculated texture complexity; Based on the following formula, Select the DCT block size: ； ； in, and These are the preset first texture complexity threshold and the second texture complexity threshold, respectively. This indicates the size of the selected DCT block.

7. A variable-granularity adaptive coding system for smoke and dust hazard environments, characterized in that, The system includes: a dust concentration sensor, a visibility observer, a transmittance compensation coefficient calculation module, an image acquisition and compensation module, a visible light camera, a SWIR camera, a feature extraction and feature fusion module, a DCT transformation module, a variable granularity encoded dust suppression module, and a DCT quantization module. Dust concentration sensor is used to obtain the current dust concentration and current dust particle size in a smoke and dust disaster environment. Visibility monitoring equipment is used to obtain the current visibility in smoke and dust disaster environments; The transmittance compensation coefficient calculation module is used to detect whether the dust concentration is greater than the preset first dust concentration threshold and whether the visibility is less than the preset first visibility threshold. When the dust concentration is greater than the preset first dust concentration threshold or the visibility is less than the preset first visibility threshold, dynamic compensation of dust infrared transmittance is performed based on dust concentration and visibility to calculate the transmittance compensation coefficient. The image acquisition and compensation module is used to simultaneously start the visible light camera and the SWIR camera to capture images of the smoke and dust disaster environment scene, obtain visible light images and SWIR images of the smoke and dust disaster environment scene, and perform gain compensation on the SWIR image based on the transmittance compensation coefficient to obtain the compensated SWIR image. The feature extraction and feature fusion module is used to extract features from the visible light image and the compensated SWIR image respectively, to obtain visible light features and SWIR features, and to perform feature fusion based on attention mechanism and band weight matrix on the visible light features and SWIR features to obtain fused features and restore the fused image. The DCT transform module is used to select the DCT block size by combining the local texture complexity based on Zernike moments, and to perform DCT transform on the fused image based on the selected DCT block size to obtain a set of DCT coefficients. A variable granularity encoded dust suppression module is used to detect abnormal high-frequency coefficients based on the mapping relationship between dust particle size and DCT coefficients, and to apply quantization parameter threshold constraints to abnormal high-frequency coefficients. The DCT quantization module is used to perform variable-granularity DCT quantization on the fused image based on quantization parameters and quantization parameter threshold constraints to obtain the entropy encoding of the smoke and dust disaster environment scene. Abnormal high-frequency coefficients are detected based on the mapping relationship between dust particle size and DCT coefficients. A threshold constraint of quantization parameters is applied to the abnormal high-frequency coefficients, including: Based on the mapping relationship between dust particle size and DCT coefficient, the threshold for abnormal high frequency is determined; Each DCT coefficient is compared with an abnormal high-frequency threshold. DCT coefficients that are less than the abnormal high-frequency threshold are identified as normal coefficients, and DCT coefficients that are greater than or equal to the abnormal high-frequency threshold are identified as abnormal high-frequency coefficients. For normal coefficients, assign preset conventional quantization parameters; For abnormally high-frequency coefficients, a threshold constraint on the quantization parameters is applied on the basis of the regular quantization parameters to obtain the constrained quantization parameters, which are then assigned to the abnormally high-frequency coefficients.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform a variable granularity adaptive coding method for smoke and dust hazard environments as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement a variable granularity adaptive coding method for smoke and dust disaster environments as described in any one of claims 1 to 6.

Images