Machine patrol image transmission method based on image compression algorithm model

Abstract

The invention discloses a machine patrol image transmission method based on an image compression algorithm model. The method comprises the following specific steps: obtaining an original patrol image; intelligently identifying the original inspection image based on an image compression algorithm model to obtain a key frame of the original inspection image, and then performing high-magnification compression on the key frame to obtain a first video frame; converting the first video frame into a bit stream to obtain a compressed video stream, and transmitting the compressed video stream to a cloud server by adopting 5G communication; decoding the compressed video stream, and restoring to obtain a first video stream; constructing an image super-resolution reconstruction network, inputting the first video stream into the image super-resolution reconstruction network, and outputting the image super-resolution reconstruction network to obtain a second video stream; and performing stream pushing on the second video stream based on cloud server stream pushing to obtain an unmanned aerial vehicle inspection video stream. According to the invention, intelligent identification and high-quality compression of the unmanned aerial vehicle patrol image are realized based on the image compression algorithm model, the traffic of unmanned aerial vehicle image transmission is reduced, and the transmission efficiency of the unmanned aerial vehicle patrol image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone patrol image data processing, and in particular to a drone patrol image transmission method based on an image compression algorithm model. Background Art

[0002] A drone is an advanced unmanned self-propelled aerial vehicle. In modern power systems, drone power inspection technology has become an important means to improve inspection efficiency and ensure the safe operation of power grids. Most inspection drones are equipped with high-definition cameras, infrared cameras, GPS positioning systems, remote controls and other equipment to ensure that the drones can smoothly collect data during flight. Drones can quickly patrol transmission lines, collect data on power facilities in multiple dimensions, and conduct all-round rapid video and fault monitoring of overhead line towers, brackets, conductors, insulators, shock-absorbing hammers, tension clamps, suspension clamps and other facilities, and automatically collect images such as pictures, videos, infrared thermal images and other image data during flight.

[0003] A large amount of data collected by drones needs to be transmitted back to the ground control station in real time, so that the staff can monitor the power system based on the pictures, videos, infrared thermal images and other image data collected by the drones. Therefore, it is necessary to use the drone patrol image transmission method, but most drone patrol image transmission methods cannot quickly identify inspection images and thus cannot guarantee high-quality compressed inspection images, resulting in a slow transmission speed when transmitting drone patrol image data and a large amount of transmission traffic required for transmission, which affects its transmission efficiency. Summary of the invention

[0004] The present invention provides an unmanned aerial patrol image transmission method based on an image compression algorithm model, which realizes intelligent recognition and high-quality compression of unmanned aerial patrol images based on the image compression algorithm model, reduces the traffic of unmanned aerial vehicle image transmission, and improves the transmission efficiency of unmanned aerial patrol images.

[0005] The technical solution adopted by the present invention is:

[0006] A method for transmitting an image of an aircraft patrol based on an image compression algorithm model comprises the following specific steps:

[0007] S1. Obtaining an original inspection image; wherein the original inspection image includes a first inspection image collected by the drone inspection and a second inspection image stored in the drone nest;

[0008] S2. Intelligently identify the original inspection image based on the image compression algorithm model to obtain a key frame of the original inspection image, and then perform high-rate compression on the key frame to obtain a first video frame;

[0009] S3, converting the first video frame into a bit stream to obtain a compressed video stream, and transmitting the compressed video stream to a cloud server using 5G communication;

[0010] S4. When the intranet terminal device accesses the cloud server, it selects the compressed video stream to be viewed, decodes the compressed video stream, and restores the first video stream;

[0011] S5, constructing an image super-resolution reconstruction network, inputting the first video stream into the image super-resolution reconstruction network to output a second video stream, and completing the restoration of the first video stream from low resolution to high resolution;

[0012] S6. Push the second video stream based on the cloud server to obtain the drone inspection video stream, that is, the original inspection image.

[0013] Furthermore, the intelligent recognition of the original inspection image in step S2 specifically includes the following steps:

[0014] S21, obtaining an image pixel value of the second patrol image, and predicting an image pixel value of a key frame of the first patrol image based on the image pixel value of the second patrol image, wherein the image pixel value of the second patrol image is recorded as the first image pixel value, and the image pixel value of the key frame of the first patrol image is recorded as the second image pixel value;

[0015] S22, calculate the prediction error value between the first image pixel value and the second image pixel value, and the calculation formula is as follows:

[0016] E=X-X1(1)

[0017] In formula (1), E is the prediction error value between the first image pixel value and the second image pixel value; X is the image pixel value of the second inspection image; X1 is the image pixel value of the key frame of the first inspection image;

[0018] S23, encoding the prediction error value to obtain a prediction code, wherein the encoding formula is as follows:

[0019] C E ={0,1} p (2)

[0020] In formula (2), C E is predictive coding; p is the frequency of occurrence of the pixel value of the second image;

[0021] S24, based on the first image pixel value and the conversion coefficient, calculate the transformation value required for the second image pixel value conversion, and the calculation formula is as follows:

[0022]

[0023] in,

[0024]

[0025] In formulas (3) to (4), X(m,n) is the first image pixel value, wherein m is the width of the first image pixel value; n is the height of the first image pixel value; Y is the conversion coefficient of the first image pixel value; max(X) is the maximum value of the first image pixel value; min(X) is the minimum value of the first image pixel value;

[0026] S25, respectively calculate the lossy compression value and the lossless compression value of the second image pixel value, and the specific calculation formula is as follows:

[0027]

[0028] In equations (5) to (6), L1 is the lossy compressed value of the second image pixel value; X(m,n) is the first image pixel value; X(m,n) min is the minimum possible value of the first image pixel value; S is the preset compression quantization value; L2 is the lossless compression value of the second image pixel value; p is the occurrence frequency of the second image pixel value; i is the length of Huffman coding in the image compression algorithm model.

[0029] S26, based on the predictive coding, the transformation value of the second image pixel value, the lossy compression value and the lossless compression value, the image enhancement intelligent recognition rate is calculated, and the calculation formula is as follows:

[0030]

[0031] In formula (7), C IE is the image enhancement intelligent recognition rate; L1 is the lossy compression value of the second image pixel value; L2 is the lossless compression value of the second image pixel value; X(m,n) is the first image pixel value; C E For predictive coding;

[0032] S27. Based on the image enhancement intelligent recognition rate, a key frame of the original inspection image is obtained.

[0033] Furthermore, the step S4 specifically includes the following steps:

[0034] S41, constructing an image restoration model that matches the image compression algorithm model;

[0035] S42, identifying and extracting image features in the compressed video stream based on the image restoration model and the image enhancement model to obtain a first video stream, so as to accurately encode and restore the compressed video stream into the first video stream;

[0036] The image restoration model includes an encoder-decoder structure, wherein the encoder is responsible for compressing input data into a context vector, and the decoder generates a target output sequence from the vector.

[0037] Furthermore, the image compression algorithm model includes a convolutional neural network module, which compresses and extracts features of the original inspection image layer by layer through a multi-layer network structure, and each layer gradually reduces the dimension of the data to achieve high-rate compression.

[0038] Furthermore, an image compression SDK module is integrated on the side of the drone nest; and an image compression program is installed on the intranet terminal device.

[0039] Furthermore, the cloud server streaming is based on the OBS streaming tool. Before streaming, the power user needs to configure service parameters, input the streaming address and authentication information, set up a stable network environment, and then the OBS streaming tool streams the second video stream in the cloud server.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1) The image enhancement intelligent recognition rate is calculated based on predictive coding, transformation value, lossy compression value and lossless compression value to realize intelligent recognition of the original inspection image, avoiding the removal of sensitive information during subsequent compression, resulting in a decrease in compression ratio and affecting decoding clarity;

[0042] 2) Based on the image compression algorithm model, the key frames of the original inspection images are compressed at a high rate to achieve a higher compression ratio, so that the compressed volume and the required storage space are reduced, the burden of network transmission is reduced, and the transmission efficiency of machine inspection images is improved;

[0043] 3) Select and decode compressed video streams on demand to save storage space on intranet terminal devices, effectively alleviating the image storage pressure on the platform;

[0044] 4) The image resolution of the aircraft patrol image during the conversion is improved by constructing a super-resolution reconstruction network, and the first video stream is restored from low resolution to high resolution to obtain a second video stream;

[0045] 5) By combining the image compression SDK integrated on the drone nest side and the image compression program installed on the intranet terminal device, high-quality compression of the drone patrol images can be achieved, the transmission traffic of the drone patrol images can be reduced, and the clarity of key equipment and key targets can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0047] Figure 1 It is a flow chart of a method for transmitting machine patrol images based on an image compression algorithm model of the present invention. DETAILED DESCRIPTION

[0048] Example

[0049] like Figure 1 As shown, a method for transmitting an image of an aircraft patrol based on an image compression algorithm model includes the following specific steps:

[0050] S1. Obtaining an original inspection image; wherein the original inspection image includes a first inspection image collected by the drone inspection and a second inspection image stored in the drone nest;

[0051] S2. Intelligently identify the original inspection image based on the image compression algorithm model to obtain a key frame of the original inspection image, and then perform high-rate compression on the key frame to obtain a first video frame;

[0052] Among them, the intelligent recognition of the original inspection image specifically includes the following steps:

[0053] S21, obtaining an image pixel value of the second patrol image, and predicting an image pixel value of a key frame of the first patrol image based on the image pixel value of the second patrol image, wherein the image pixel value of the second patrol image is recorded as the first image pixel value, and the image pixel value of the key frame of the first patrol image is recorded as the second image pixel value;

[0054] S22, calculate the prediction error value between the first image pixel value and the second image pixel value, and the calculation formula is as follows:

[0055] E=X-X1(1)

[0056] In formula (1), E is the prediction error value between the first image pixel value and the second image pixel value; X is the image pixel value of the second inspection image; X1 is the image pixel value of the key frame of the first inspection image;

[0057] S23, encoding the prediction error value to obtain a prediction code, wherein the encoding formula is as follows:

[0058] C E ={0,1} p (2)

[0059] In formula (2), C E is predictive coding; p is the frequency of occurrence of the pixel value of the second image;

[0060] S24, based on the first image pixel value and the conversion coefficient, calculate the transformation value required for the second image pixel value conversion, and the calculation formula is as follows:

[0061]

[0062] in,

[0063]

[0064] In formulas (3) to (4), X(m,n) is the first image pixel value, wherein m is the width of the first image pixel value; n is the height of the first image pixel value; Y is the conversion coefficient of the first image pixel value; max(X) is the maximum value of the first image pixel value; min(X) is the minimum value of the first image pixel value;

[0065] S25, respectively calculate the lossy compression value and the lossless compression value of the second image pixel value, and the specific calculation formula is as follows:

[0066]

[0067] In equations (5) to (6), L1 is the lossy compressed value of the second image pixel value; X(m,n) is the first image pixel value; X(m,n) min is the minimum possible value of the first image pixel value; S is the preset compression quantization value; L2 is the lossless compression value of the second image pixel value; p is the occurrence frequency of the second image pixel value; i is the length of Huffman coding in the image compression algorithm model.

[0068] S26, based on the predictive coding, the transformation value of the second image pixel value, the lossy compression value and the lossless compression value, the image enhancement intelligent recognition rate is calculated, and the calculation formula is as follows:

[0069]

[0070] In formula (7), C IE is the image enhancement intelligent recognition rate; L1 is the lossy compression value of the second image pixel value; L2 is the lossless compression value of the second image pixel value; X(m,n) is the first image pixel value; C E For predictive coding;

[0071] S27, based on the image enhancement intelligent recognition rate, obtaining a key frame of the original inspection image;

[0072] S3, converting the first video frame into a bit stream to obtain a compressed video stream, and transmitting the compressed video stream to a cloud server using 5G communication;

[0073] S4. When the intranet terminal device accesses the cloud server, it selects the compressed video stream to be viewed, decodes the compressed video stream, and restores the first video stream;

[0074] S5, constructing an image super-resolution reconstruction network, inputting the first video stream into the image super-resolution reconstruction network to output a second video stream, and completing the restoration of the first video stream from low resolution to high resolution;

[0075] S6. Push the second video stream based on the cloud server to obtain the drone inspection video stream, that is, the original inspection image.

[0076] Furthermore, in step S2

[0077] Furthermore, the step S4 specifically includes the following steps:

[0078] S41, constructing an image restoration model that matches the image compression algorithm model;

[0079] S42, identifying and extracting image features in the compressed video stream based on the image restoration model and the image enhancement model to obtain a first video stream, so as to accurately encode and restore the compressed video stream into the first video stream;

[0080] The image restoration model includes an encoder-decoder structure, wherein the encoder is responsible for compressing input data into a context vector, and the decoder generates a target output sequence from the vector.

[0081] A further technical solution of this embodiment is that the image compression algorithm model includes a convolutional neural network module, and the convolutional neural network module compresses and extracts features of the original inspection image layer by layer through a multi-layer network structure, and each layer gradually reduces the dimension of the data to achieve high-rate compression.

[0082] According to a further technical solution of this embodiment, an image compression SDK module is integrated on the side of the drone nest; an image compression program is installed on the intranet terminal device, and high-quality compression of the drone patrol image is achieved through asymmetric technology development.

[0083] A further technical solution of this embodiment is that the cloud server streaming is based on the OBS streaming tool. Before streaming, the power user needs to configure the service parameters, input the streaming address and authentication information, set up a stable network environment, and then the OBS streaming tool streams the second video stream in the cloud server.

[0084] The method for transmitting patrol images of the present invention calculates the image enhancement intelligent recognition rate based on predictive coding, transformation value, lossy compression value and lossless compression value, realizes intelligent recognition of original patrol images, avoids removal of sensitive information during subsequent compression, causes compression ratio to drop, and affects decoding clarity; based on the image compression algorithm model, the key frames of the original patrol images are compressed at a high rate to achieve a higher compression ratio, so that the compressed volume and the required storage space become smaller, reduce the burden of network transmission, and improve the transmission efficiency of patrol images; selects and decodes and restores compressed video streams on demand to save storage space of intranet terminal devices, and effectively alleviates the platform image storage pressure; improves the image resolution of patrol images during conversion through the constructed super-resolution reconstruction network, completes the restoration of the first video stream from low resolution to high resolution to obtain the second video stream; by combining the image compression SDK integrated on the drone nest side and the image compression program installed on the intranet terminal device, high-quality compression of patrol images is achieved, the transmission flow of patrol images is reduced, and the clarity of key equipment and key targets can be guaranteed.

[0085] The embodiments described above are merely descriptions of preferred implementation modes of the present invention and are not intended to limit the scope of the present invention. Without departing from the principles and essence of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for transmitting patrol images based on an image compression algorithm model, characterized in that: The specific steps include: S1. Obtaining an original inspection image; wherein the original inspection image includes a first inspection image collected by the drone inspection and a second inspection image stored in the drone nest; S2. Intelligently identify the original inspection image based on the image compression algorithm model to obtain a key frame of the original inspection image, and then perform high-rate compression on the key frame to obtain a first video frame; S3, converting the first video frame into a bit stream to obtain a compressed video stream, and transmitting the compressed video stream to a cloud server using 5G communication; S4. When the intranet terminal device accesses the cloud server, it selects the compressed video stream to be viewed, decodes the compressed video stream, and restores the first video stream; S5, constructing an image super-resolution reconstruction network, inputting the first video stream into the image super-resolution reconstruction network to output a second video stream, and completing the restoration of the first video stream from low resolution to high resolution; S6. Push the second video stream based on the cloud server to obtain the drone inspection video stream, that is, the original inspection image.

2. According to the method of transmitting patrol images based on an image compression algorithm model according to claim 1, it is characterized in that: The intelligent recognition of the original inspection image in step S2 specifically includes the following steps: S21, obtaining an image pixel value of the second patrol image, and predicting an image pixel value of a key frame of the first patrol image based on the image pixel value of the second patrol image, wherein the image pixel value of the second patrol image is recorded as the first image pixel value, and the image pixel value of the key frame of the first patrol image is recorded as the second image pixel value; S22, calculate the prediction error value between the first image pixel value and the second image pixel value, and the calculation formula is as follows: E=X-X1(1) In formula (1), E is the prediction error value between the first image pixel value and the second image pixel value; X is the image pixel value of the second inspection image; X1 is the image pixel value of the key frame of the first inspection image; S23, encoding the prediction error value to obtain a prediction code, wherein the encoding formula is as follows: C E ＝{0，1} p (2) In formula (2), C E is predictive coding; p is the frequency of occurrence of the pixel value of the second image; S24, based on the first image pixel value and the conversion coefficient, calculate the transformation value required for the second image pixel value conversion, and the calculation formula is as follows: in, In formulas (3) to (4), X(m,n) is the first image pixel value, wherein m is the width of the first image pixel value; n is the height of the first image pixel value; Y is the conversion coefficient of the first image pixel value; max(X) is the maximum value of the first image pixel value; min(X) is the minimum value of the first image pixel value; S25, respectively calculate the lossy compression value and the lossless compression value of the second image pixel value, and the specific calculation formula is as follows: In equations (5) to (6), L1 is the lossy compressed value of the second image pixel value; X(m,n) is the first image pixel value; X(m,n) min is the minimum possible value of the first image pixel value; S is the preset compression quantization value; L2 is the lossless compression value of the second image pixel value; p is the occurrence frequency of the second image pixel value; i is the length of Huffman coding in the image compression algorithm model. S26, based on the predictive coding, the transformation value of the second image pixel value, the lossy compression value and the lossless compression value, the image enhancement intelligent recognition rate is calculated, and the calculation formula is as follows: In formula (7), C IE is the image enhancement intelligent recognition rate; L1 is the lossy compression value of the second image pixel value; L2 is the lossless compression value of the second image pixel value; X(m,n) is the first image pixel value; C E For predictive coding; S27. Based on the image enhancement intelligent recognition rate, a key frame of the original inspection image is obtained.

3. The method for transmitting patrol images based on an image compression algorithm model according to claim 1, characterized in that: The step S4 specifically comprises the following steps: S41, constructing an image restoration model that matches the image compression algorithm model; S42, identifying and extracting image features in the compressed video stream based on the image restoration model and the image enhancement model to obtain a first video stream, so as to accurately encode and restore the compressed video stream into the first video stream; The image restoration model includes an encoder-decoder structure, wherein the encoder is responsible for compressing input data into a context vector, and the decoder generates a target output sequence from the vector.

4. The method for real-time transmission of aircraft patrol images based on image compression according to claim 1, characterized in that: The image compression algorithm model includes a convolutional neural network module, which compresses and extracts features of the original inspection image layer by layer through a multi-layer network structure, and each layer gradually reduces the dimension of the data to achieve high-rate compression.

5. The method for transmitting patrol images based on an image compression algorithm model according to claim 1, characterized in that: An image compression SDK module is integrated on the side of the drone nest; an image compression program is installed on the intranet terminal device.

6. The method for real-time transmission of aircraft patrol images based on image compression according to claim 1, characterized in that: The cloud server streaming is based on the OBS streaming tool. Before streaming, the power user needs to configure service parameters, input the streaming address and authentication information, set a stable network environment, and then the OBS streaming tool streams the second video stream in the cloud server.

Images