Method and apparatus for partial modification of jpeg images

Abstract

The application provides a JPEG image partial modification method and device, which comprises the following steps: receiving data from a JPEG data stream, processing the data bit by bit, and reading the data into a 256-byte input buffer in sequence; receiving a JPEG header, decoding minimum coding units (MCU) of the image in sequence according to parameters read from the header, and determining whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are in a specified image region to be modified; if the current MCU needs to be modified, decoding the current MCU into YUV data, obtaining a character dot matrix corresponding to the current MCU from a character library, superimposing the character dot matrix onto the YUV data, re-encoding the YUV data superimposed with the character dot matrix, and writing the YUV data into an output buffer; and if the current MCU does not need to be modified, directly writing the current MCU into the output buffer. Through the application, image data can be quickly and stably modified and transmitted, and data loss caused by image data delay and loss is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a JPEG image partial modification method and device. BACKGROUND

[0002] With the development of the Internet of Things technology, the number of Internet of Things sensing devices such as image sensors has increased explosively, thereby generating a large amount of real-time image data. The real-time image data generated by a large number of camera devices often needs to add the names of collection points and various customized information, and therefore, an image processing device needs to be added between the image sensing device and the edge server for modifying the image data from the image sensing device and adding the display of various customized information. The conventional modification method uses a full decoding method, which is very inefficient in resource-limited Internet of Things devices, resulting in a large amount of image data that cannot be processed in time or even lost or damaged. In the patent "Method and device for modifying a local image of a JPEG image", a method for modifying a JPEG local image is proposed, but this method is limited to the case where the processing area is located at the lower left corner or the lower right corner of the full image. Some scenes have higher requirements for the stability and randomness of image data processing, such as real-time monitoring of water levels in the field of hydrology and emergency alarm information for floods that need to be processed in real time. Such data requires real-time modification at the upper left corner of the collected image data and at any position, and has the characteristic that the amount of modified data accounts for a small proportion of the entire image. Therefore, higher requirements are put forward for the speed and efficiency of image data modification. SUMMARY

[0003] To solve at least one technical problem in the prior art, the present application provides a JPEG image partial modification method and device.

[0004] In a first aspect, the present application provides a JPEG image partial modification method, which comprises:

[0005] receiving data from a JPEG data stream, processing it bit by bit, and reading the data into a 256-byte input buffer in sequence;

[0006] receiving a JPEG header, decoding the minimum coding unit (MCU) of the image in sequence according to the parameters read by the header, and determining whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within the specified image region to be modified;

[0007] if the current MCU needs to be modified, decoding the current MCU into YUV data, obtaining the text dot matrix corresponding to the current MCU from the font library and superimposing it onto the YUV data, and encoding the YUV data superimposed with the text dot matrix again and writing it into the output buffer;

[0008] If the current MCU does not need to be modified, the current MCU is directly written into the output buffer.

[0009] Optionally, after receiving the JPEG header, the method further comprises:

[0010] receiving image identification information, defining a quantization table, image basic information, and defining a Huffman table from the JPEG header;

[0011] determining whether the JPEG encoding format is a JFIF format according to the image identification information, and determining the number of Y, Cb, and Cr components according to the image basic information;

[0012] After the header is parsed, the header data received above is synchronously written into the output buffer.

[0013] Optionally, if the current MCU does not need to be modified, the step of writing the current MCU directly into the output buffer comprises:

[0014] S41. If the current MCU does not need to be modified, starting from the start address of the current MCU color component unit data stream, bit data is read in sequence until the read bit data string matches the encoding in the direct current Huffman table of the component obtained from the header, and the corresponding direct current component value is obtained by querying the direct current Huffman table;

[0015] S42. Continue reading data by bit until the read encoding matches the encoding of the alternating current Huffman table of the component, and the corresponding alternating current component value is obtained by querying the alternating current Huffman table;

[0016] S43. Step S42 is repeatedly executed until the condition for ending the alternating current component data is met, and the condition is met as long as one of the following conditions is met: the alternating current component value read is zero, indicating that the alternating current variable after that is all zero; or all 63 alternating current components have been read;

[0017] S44. Steps S41-S43 are repeatedly executed until all the MCUs of the current image block are read, and the current MCU is written into the output buffer.

[0018] Optionally, if the current MCU needs to be modified, the current MCU is decoded into YUV data, the corresponding text dot matrix of the current MCU is obtained from the font library and superimposed on the YUV data, the YUV data superimposed with the text dot matrix is encoded again and written into the output buffer.

[0019] S31. If the current MCU needs to be modified, reading starts from the start of the current MCU color component unit data stream by bit, and the reading step is consistent with S41-S43;

[0020] S32. The difference direct current components of each color component unit are sequentially decoded according to the formula C n = DCn-1 +Diff is the difference correction variable, and Diff is the actual direct current variable correction of the previous color component unit to the current direct current variable;

[0021] S33. The quantization table decoded from the color component information field in the mark section SOF is used to dequantize different color components, and the 64 values of the 8x8 color component unit are multiplied by the values at the same position in the corresponding quantization table one by one during dequantization;

[0022] S34. Each 8x8 color component unit after dequantization is Zig-zag encoded according to the element number;

[0023] S35. The inverse discrete cosine transform is performed to convert the numerical values of the color component unit matrix from the frequency domain to the time-space domain, and the size of the time-space domain matrix remains unchanged after the inverse discrete cosine transform;

[0024] S36. The MCU decoding obtains a 16x16 pixel image in YUV format, and a 16-bit dot matrix font character is selected to modify the data of a single image block. The character data is divided into two categories: half-width and full-width, occupying 16x8 and 16x16 pixels respectively. According to the original text dot matrix, the character data is mapped to the continuous image block in sequence, and according to the brightness of the block pixels and the complementary color principle, the formula is used to determine the brightness value of the superimposed text to increase the contrast between the superimposed text and the background color, so that the text is clearly displayed, wherein luminance is the obtained brightness value, the image is modified and the original text arrangement is maintained;

[0025] S37. The end-of-sequence text character is processed for direct current component recovery. For the last block of the continuous modified block, the direct current component is set to the difference prefix of the subsequent block, and the result is saved;

[0026] S38. The modified image block is processed by the encoding module to complete the processes of discrete cosine transform, Zig-zag encoding, quantization, difference encoding of direct current coefficient, and Huffman encoding, forming new MCU data;

[0027] S39. Until the encoding of all data units of the image block is completed, the new MCU is written into the output buffer.

[0028] Optionally, the JPEG image part modification method further comprises:

[0029] S51. Data is sequentially read from the output buffer and assembled into a packet for transmission to the gateway;

[0030] S52. Step S51 is repeatedly performed until the image data processing is completed and the complete JPEG image data is assembled.

[0031] In a second aspect, the present application provides a JPEG image partial modification device, comprising:

[0032] An input module, configured to receive data from a JPEG data stream, process the data bit by bit, and read the data into a 256-byte input buffer sequentially;

[0033] A determination module, configured to receive a JPEG header, decode a minimum coding unit (MCU) of an image according to parameters read from the header, and determine whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within a designated image region to be modified;

[0034] A first processing module, configured to, if the current MCU needs to be modified, decode the current MCU into YUV data, obtain a character dot matrix corresponding to the current MCU from a font library, superimpose the character dot matrix onto the YUV data, encode the YUV data with the superimposed character dot matrix again, and write the encoded YUV data into an output buffer;

[0035] A second processing module, configured to, if the current MCU does not need to be modified, write the current MCU directly into the output buffer.

[0036] Optionally, the JPEG image partial modification device further comprises an analysis module, configured to:

[0037] receive image identification information, a definition quantization table, image basic information, and a definition Huffman table from the JPEG header;

[0038] determine whether the JPEG coding format is a JFIF format according to the image identification information, and determine the number of Y, Cb, and Cr components according to the image basic information;

[0039] write the received header data into the output buffer synchronously after the header analysis is completed.

[0040] Optionally, the second processing module is configured to:

[0041] S41. If the current MCU does not need to be modified, read bit data from a start address of a color component unit data stream of the current MCU sequentially until the read bit data string matches the coding in a direct current Huffman table of the component obtained from the header, and query the direct current Huffman table to obtain a corresponding direct current component value;

[0042] S42. Continue to read data bit by bit until the read coding matches the coding in an alternating current Huffman table of the component, and query the alternating current Huffman table to obtain a corresponding alternating current component value;

[0043] S43. Keep repeating step S42 until the end condition of AC component data is met, the end condition has two, one of which can be met: the AC component value is read in zero, indicating that the AC variable is all zero after; has read all 63 AC components;

[0044] S44. Keep repeating S41-S43 until all MCU of the current image block is read, write the current MCU to the output buffer.

[0045] Optionally, the first processing module is used for:

[0046] S31. If the current MCU needs to be modified, read from the start of the current MCU color component unit data stream, the reading step is consistent with S41-S43;

[0047] S32. Decode the difference DC component of each color component unit in turn, from the formula C n = DC n-1 + Diff, the current DC variable is corrected by the actual DC variable of the previous color component unit, wherein Diff is the difference correction variable;

[0048] S33. Use the quantization table decoded from the color component information field in the mark section SOF to dequantize different color components, and multiply the 64 values of the 8x8 color component unit by the values in the corresponding quantization table in the same position during the dequantization process;

[0049] S34. Each 8x8 color component unit after dequantization is Zig-zag encoded according to the element number;

[0050] S35. Perform inverse discrete cosine transform to convert the color component unit matrix value from the frequency domain to the time-space domain, and the size of the time-space domain matrix remains unchanged after the inverse discrete cosine transform;

[0051] S36. The MCU is decoded to obtain a 16x16 pixel image in YUV format, and a single image block is modified by selecting a 16-bit dot matrix character library. The character data is divided into two categories: half-angle and full-angle, which occupy 16x8 and 16x16 pixels respectively. According to the original text dot matrix, it is mapped to a continuous image block in turn, and according to the brightness and complementary color principle of the block pixels, the formula is used to determine the brightness value of the superimposed text to increase the contrast between the superimposed text and the background color, so that the text is clearly displayed, wherein luminance is the obtained brightness value, the image is modified and the original text arrangement is maintained;

[0052] S37. The DC component recovery process is performed on the end-of-sequence character, and the DC component is set to the difference prefix of the subsequent block for the last block of the continuous modified block, and the result is saved;

[0053] S38. The modified image block is processed by the encoding module to complete the processes of discrete cosine transform, Zig-zag encoding, quantization, difference encoding of the DC coefficient, and Huffman encoding, thereby forming new MCU data;

[0054] S39. Until the encoding of all data units of the current image block is completed, the new MCU is written into the output buffer.

[0055] Optionally, the JPEG image partial modification device further comprises an output module, configured to:

[0056] S51. Data is sequentially read from the output buffer, and the data is assembled into a packet and sent to the gateway;

[0057] S52. Step S51 is repeatedly performed until the image data processing is completed and the image data is assembled into a complete JPEG image data.

[0058] In the present application, data from the JPEG data stream is processed bit by bit, and the data is sequentially read into a 256-byte input buffer; the JPEG header is received, and the minimum coding unit MCU of the image is sequentially decoded according to the parameters read by the header, and it is determined whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are in the specified image region to be modified; if the current MCU needs to be modified, the current MCU is decoded into YUV data, the current MCU corresponding to the text dot matrix is obtained from the font library and superimposed on the YUV data, and the YUV data superimposed with the text dot matrix is encoded again and written into the output buffer; if the current MCU does not need to be modified, the current MCU is directly written into the output buffer. Through the present application, the image data can be quickly and stably modified and transmitted, and the data loss caused by image data delay and loss is greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The flowchart of the first embodiment of the JPEG image partial modification method of the present application is shown in the figure;

[0060] Figure 2 The frame diagram of an embodiment of the JPEG image partial modification method of the present application is shown in the figure;

[0061] Figure 3 The flowchart of the second embodiment of the JPEG image partial modification method of the present application is shown in the figure;

[0062] Figure 4 The hierarchical structure diagram of the JPEG image used in the embodiment of the present application is shown in the figure;

[0063] Figure 5 Figure 1 is a schematic diagram of functional modules of an embodiment of the JPEG image partial modification device of the present application.

[0064] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0066] In a first aspect, an embodiment of the present application provides a JPEG image partial modification method.

[0067] In an embodiment, the JPEG image partial modification method comprises the following steps: Figure 1 Figure 1 Figure 2 is a schematic diagram of a flowchart of an embodiment of the JPEG image partial modification method of the present application. As shown in Figure 2, the JPEG image partial modification method comprises the following steps: Figure 1

[0068] In step S10, data from the JPEG data stream is received and processed bit by bit, and the data is read into a 256-byte input buffer in sequence.

[0069] In this embodiment, the data enters the input buffer by bytes. A 256-byte input buffer is used, and the decoding module reads the data in the buffer bit by bit and judges whether the buffer is read out completely. When the data in the input buffer is read out completely, the decoding module notifies the upstream device to transmit 256-byte data into the input buffer again, and the above steps are repeated until the image data processing is completed.

[0070] In step S20, the JPEG header is received, and the minimum coding unit (MCU) of the image is decoded in sequence according to the parameters read from the header, and it is determined whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within the specified image region to be modified.

[0071] In this embodiment, the data receives the JPEG header under the control of the decoding module, decodes the MCUs of the image in sequence according to the parameters read from the header, and determines whether the MCUs need to be modified by comparing whether the coordinates of the current MCU are within the specified image region to be modified. That is, when the coordinates of the current MCU are within the specified image region to be modified, it is determined that the MCU needs to be modified; and when the coordinates of the current MCU are not within the specified image region to be modified, it is determined that the MCU does not need to be modified.

[0072] ​​Step S30, if the current MCU needs to be modified, the current MCU is decoded into YUV data, the current MCU corresponding text dot matrix is obtained from the font library and superimposed into the YUV data, and the YUV data superimposed with the text dot matrix is encoded again and written into the output buffer;

[0073] Step S40, if the current MCU does not need to be modified, the current MCU is directly written into the output buffer.

[0074] In the embodiment, reference is made to Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a framework of an embodiment of the JPEG image partial modification method. Figure 2 As shown in the figure, the header analysis part of the encoding and decoding module receives data from the input buffer by byte, and processes it by bit. Specifically:

[0075] APP0 (image identification information), DQT (definition quantization table), SOF0 (image basic information), DHT (definition Huffman table) and other parts are received from the JPEG header;

[0076] It is determined whether the JPEG encoding format is JFIF format according to APP0, and the number of Y, Cb and Cr components is determined according to SOF0;

[0077] After the header analysis is completed, the above-mentioned received header data is written into the output buffer synchronously.

[0078] The structure of the image data stream is analyzed in the order from macro to micro, that is, data stream -> MCU -> Y, Cb and Cr color components. In the data stream, the arrangement method of the MCU is from left to right and from top to bottom. Each MCU is divided into a plurality of 8x8 color classification units. Then the MCU is processed in turn, and the specific conditions are classified as follows:

[0079] If the decoding module monitors that the current MCU is an image block that does not need to be modified, the current MCU is read; after the MCU reading is completed, it is directly written into the output buffer;

[0080] If the decoding module monitors that the current MCU is an image block that needs to be modified, the current MCU is decoded; after the MCU decoding is completed, it is superimposed into the decoded YUV block data. After the image block modification is completed, the encoding module re-encodes the MCU according to the original format and writes it into the output buffer.

[0081] If the decoding module monitors that the current MCU is an image block that does not need to be modified, the decoding module reads the current MCU data according to the read Huffman table data, and the specific steps are as follows:

[0082] S41. If the current MCU does not need to be modified, read the bits sequentially from the start address of the current MCU color component unit data stream, until the read bit data string code matches the code in the DC Huffman table of the component obtained in the header, and query the DC Huffman table to obtain the corresponding DC component value;

[0083] S42. Continue reading the data by bits until the read code matches the code of the AC Huffman table of the component, and query the AC Huffman table to obtain the corresponding AC component value;

[0084] S43. Repeat step S42 until the end condition of the AC component data is met, and the end condition has two conditions, one of which can be met: the AC component value read is zero, indicating that the subsequent AC variable is all zero; all 63 AC components have been read;

[0085] S44. Repeat S41-S43 until all MCU of the current image block is read, and write the current MCU to the output buffer.

[0086] Further, if the current MCU is an image block that needs to be modified, the decoding module decodes the current MCU and modifies the image data according to the read Huffman table data, and the specific steps are as follows:

[0087] S31. If the current MCU needs to be modified, read the bits from the start of the current MCU color component unit data stream, and the reading steps are consistent with S41-S43;

[0088] S32. Decode the difference DC component of each color component unit in turn, according to the formula C n = DC n-1 + Diff, wherein Diff is the difference correction variable, and the current DC variable is corrected by the actual DC variable of the previous color component unit;

[0089] S33. Use the quantization table decoded from the color component information field in the SOF marker for different color components for dequantization, and multiply the 64 values of the 8x8 color component unit by the values at the same position in the corresponding quantization table during dequantization;

[0090] S34. Reverse Zig-zag encode each 8x8 color component unit after dequantization according to the element number;

[0091] S35. Perform inverse discrete cosine transform to convert the color component unit matrix values from the frequency domain to the time-space domain, and the size of the time-space domain matrix remains unchanged after inverse discrete cosine transform;

[0092] S36. After the MCU is decoded, a 16x16 pixel image in YUV format is obtained, and a 16-bit dot matrix character library is selected to modify the data of a single image block. The character data is classified into two categories, i.e., half-angle and full-angle, which occupy 16x8 and 16x16 pixels, respectively. The original text dot matrix is sequentially mapped to the continuous image blocks, and the formula The luminance value of the superimposed text is determined to increase the contrast between the superimposed text and the background color, so that the text is clearly displayed. The luminance is the obtained luminance value, the image is modified, and the original text arrangement is maintained.

[0093] S37. The direct current component recovery processing is performed on the end character of the sequential text. For the last block of the continuous modified block, the direct current component is set as the differential prefix of the subsequent block, and the result is saved.

[0094] S38. The modified image block is subjected to the process of discrete cosine transform, Zig-zag coding, quantization, differential coding of direct current coefficients, and Huffman coding in the encoding module to form new MCU data.

[0095] S39. Until the encoding of all data units of the image block is completed, the new MCU is written into the output buffer.

[0096] Further, in an embodiment, the JPEG image partial modification method further comprises:

[0097] S51. The data is sequentially read from the output buffer, and the data is assembled into a packet and sent to the gateway.

[0098] S52. Step S51 is repeatedly performed until the image data processing is completed, and the image data is assembled into a complete JPEG image data.

[0099] In the embodiment, a 512-byte small buffer is used. When the buffer data is full, the data output part sequentially reads the cached data from the output buffer, assembles the data into a packet, and sends the packet to the gateway. The above steps are repeatedly performed until the image data processing is completed, and the image data is assembled into a complete JPEG image data.

[0100] In the above embodiment, the direct current component recovery processing is performed on the end character of the sequential text. Since the direct current components are stored by differential coding between MCUs, if the original block is directly connected after the image block is modified without processing, the direct current components will be misaligned, resulting in image misalignment and distortion. Therefore, for the last block of the continuous modified block, the direct current component is set as the differential prefix of the subsequent block, and the result is saved.

[0101] This embodiment targets IoT devices, enabling rapid modification of JPEG images within resource-constrained IoT devices, thereby achieving efficient image data transmission. The input buffer, MCU decoding module, YUV modification module, MCU encoding module, and output buffer involved in this embodiment can be flexibly deployed in IoT devices. Using a microcontroller as the main control device enables fast and efficient image data processing with low power consumption. Using a small input / output buffer for bit-by-bit data processing reduces memory usage to less than 5KB, minimizing microcontroller memory consumption. In practice, it has been shown to run stably on microcontrollers with 16KB of memory and an 8MHz clock speed. The JPEG fast image modification mechanism involved in this embodiment enables rapid and stable modification and transmission of image data, significantly reducing data loss caused by image data latency and loss.

[0102] In one embodiment, reference is made to Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of the JPEG image partial modification method of the present invention. Figure 3 As shown, methods for partially modifying JPEG images include:

[0103] The encoding / decoding module receives data from the JPEG data stream byte by byte, processes it bit by bit, and the FIFO reader of the encoding / decoding module reads the image data sequentially into a 256-byte small buffer.

[0104] Under the control of the decoding module, the data receives the JPEG header, decodes the MCU (Minimum Coded Unit) of the image sequentially according to the parameters read from the header, and determines whether the MCU needs to be modified by comparing whether the current coordinates of the MCU are within the specified area of ​​the image to be modified.

[0105] If the MCU needs to modify the image block, it will decode the block into YUV data;

[0106] The character dot matrix corresponding to the current block is obtained from the character library and superimposed onto the YUV data obtained by decoding the block data above;

[0107] The image block data with text dot matrix overlaid was re-encoded and written to the output buffer;

[0108] If the current MCU does not contain the image block that needs to be modified, write the block data directly to the output buffer.

[0109] Reference Figure 4 , Figure 4 This is a hierarchical structure diagram of the JPEG image used in embodiments of the present invention. For example... Figure 4As shown, the JPEG image is composed of a JPEG header and a plurality of MCUs, and the MCU structure includes Y, Cb, and Cr; the Y, Cb, and Cr structure includes a differential direct current component and n alternating current components.

[0110] In a second aspect, the embodiments of the present application further provide a JPEG image partial modification device.

[0111] In an embodiment, the JPEG image partial modification device is used for modifying a specified region of a JPEG image. Figure 5 , Figure 5 FIG. 1 is a schematic diagram of functional modules of an embodiment of the JPEG image partial modification device of the present application. As shown, the JPEG image partial modification device includes: Figure 5

[0112] An input module 10 is used for receiving data from a JPEG data stream, processing the data bit by bit, and reading the data into a 256-byte input buffer in sequence;

[0113] A determination module 20 is used for receiving a JPEG header, decoding a minimum coding unit (MCU) of an image in sequence according to parameters read from the header, and determining whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within a specified region to be modified;

[0114] A first processing module 30 is used for decoding the current MCU into YUV data if the current MCU needs to be modified, obtaining a text dot matrix corresponding to the current MCU from a font library, superimposing the text dot matrix onto the YUV data, re-encoding the YUV data superimposed with the text dot matrix, and writing the re-encoded YUV data into an output buffer;

[0115] A second processing module 40 is used for directly writing the current MCU into the output buffer if the current MCU does not need to be modified.

[0116] Further, in an embodiment, the JPEG image partial modification device further includes an analysis module used for:

[0117] receiving image identification information, a definition quantization table, image basic information, and a definition Huffman table from the JPEG header;

[0118] determining whether the JPEG encoding format is a JFIF format according to the image identification information, and determining the number of Y, Cb, and Cr components according to the image basic information;

[0119] synchronously writing the received header data into the output buffer after the header analysis is completed.

[0120] Further, in an embodiment, the second processing module 40 is used for:

[0121] ​S41. If the current MCU does not need to be modified, read the bits sequentially from the start address of the current MCU color component unit data stream, until the read bit data string code matches the code in the DC Huffman table of the component obtained in the header, and query the DC Huffman table to obtain the corresponding DC component value;

[0122] S42. Continue reading the data by bits until the read code matches the code of the AC Huffman table of the component, and query the AC Huffman table to obtain the corresponding AC component value;

[0123] S43. Repeat step S42 until the end condition of the AC component data is met, and the end condition has two conditions, one of which can be met: the read AC component value is zero, indicating that the subsequent AC variable is all zero; all 63 AC components have been read;

[0124] S44. Repeat S41-S43 until all MCU of the current image block is read, and write the current MCU to the output buffer.

[0125] Further, in an embodiment, the first processing module 30 is configured to:

[0126] S31. If the current MCU needs to be modified, read the bits from the start of the current MCU color component unit data stream, and the reading steps are consistent with S41-S43;

[0127] S32. Decode the difference DC component of each color component unit in sequence, according to the formula C n = DC n-1 + Diff, wherein Diff is the difference correction variable, and the current DC variable is obtained by correcting the actual DC variable of the previous color component unit;

[0128] S33. Dequantize the quantization table decoded from the color component information field in the mark section SOF for different color components, and multiply the 64 values of the 8x8 color component unit by the values at the same position in the corresponding quantization table during the dequantization process;

[0129] S34. Reverse Zig-zag encode each 8x8 color component unit after dequantization according to the element number;

[0130] S35. Perform inverse discrete cosine transform to convert the color component unit matrix values from the frequency domain to the time-space domain, and the size of the time-space domain matrix remains unchanged after the inverse discrete cosine transform;

[0131] S36. The MCU is decoded to obtain a 16x16 pixel image in YUV format, and a 16-bit dot matrix character library is selected to modify the data of a single image block. The character data is classified into two categories, i.e., half-angle and full-angle, which occupy 16x8 and 16x16 pixels, respectively. The original text dot matrix is sequentially mapped to the continuous image blocks, and the formula is used to modify the image and keep the original text arrangement according to the brightness of the block pixels and the principle of complementary colors. The luminance value of the superimposed text is determined to increase the contrast between the superimposed text and the background color, so that the text is clearly displayed. The luminance is the obtained luminance value, the image is modified, and the original text arrangement is kept;

[0132] S37. The direct current component recovery process is performed on the end character of the sequential text. For the last block of the continuous modified block, the direct current component is set as the difference preposition of the subsequent block, and the result is saved.

[0133] S38. The modified image block is processed by the encoding module to complete the processes of discrete cosine transform, Zig-zag encoding, quantization, difference encoding of direct current coefficients, and Huffman encoding, thereby forming new MCU data.

[0134] S39. Until the encoding of all data units of the image block is completed, the new MCU is written into the output buffer.

[0135] Further, in an embodiment, the JPEG image partial modification device further comprises an output module for:

[0136] S51. The data is sequentially read from the output buffer, and the data is assembled into a packet and sent to the gateway.

[0137] S52. Step S51 is repeatedly performed until the image data processing is completed and the complete JPEG image data is assembled.

[0138] The functions of the modules in the above-described JPEG image partial modification device correspond to the steps in the above-described embodiment of the JPEG image partial modification method. The functions and implementation processes are not repeated here.

[0139] It should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0140] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc) as described above, and includes a plurality of instructions for causing a terminal device to execute the methods described in the various embodiments of the present application.

[0142] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for partially modifying a JPEG image, characterized in that, The method for partially modifying a JPEG image includes: Receive data from the JPEG data stream, process it bit by bit, and read the data sequentially into a 256-byte input buffer; Receive the JPEG header, decode the smallest encoding unit (MCU) of the image sequentially according to the parameters read from the header, and determine whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within the specified image area to be modified. If the current MCU needs to be modified, the current MCU is decoded into YUV data, the character dot matrix corresponding to the current MCU is obtained from the character library and superimposed on the YUV data, the YUV data with superimposed character dot matrix is ​​encoded again and written to the output buffer. If the current MCU does not need to be modified, then write the current MCU directly to the output buffer; The steps of decoding the current MCU into YUV data, obtaining the corresponding character dot matrix from the character library and overlaying it into the YUV data, encoding the overlaid YUV data again, and writing it into the output buffer if the current MCU needs to be modified include: S31. If the current MCU needs to be modified, start reading bit by bit from the beginning of the current MCU color component unit data stream. The reading steps are the same as S41 to S43. S32. Decode the differential DC component of each color component unit sequentially, using the formula... The current DC variable is obtained by correcting the actual DC variable of the previous color component unit, where It is a difference correction variable; S33. Use the quantization table decoded from the color component information field in the marker segment SOF to dequantize different color components. During the dequantization process, multiply each of the 64 values ​​of the 8×8 color component unit by the corresponding value in the same position in the quantization table. S34. Reverse Zig-zag encoding of each dequantized 8×8 color component unit according to its element number; S35. Perform an inverse discrete cosine transform to convert the values ​​of the color component unit matrix from the frequency domain to the spatiotemporal domain, and the size of the matrix in the spatiotemporal domain remains unchanged after the inverse discrete cosine transform. After decoding by the S36.MCU, a 16×16 pixel image in YUV format is obtained. A 16-bit dot matrix character library is selected to modify the data of a single image block. The character data is divided into two categories: half-width and full-width, occupying 16×8 and 16×16 pixels respectively. Based on the original text dot matrix, these are sequentially mapped to consecutive image blocks. Simultaneously, based on the brightness and complementary color principles of the block pixels, a formula is used... Determine the brightness value of the overlay text to increase the contrast between the overlay text and the background color, so that the text is clearly displayed, where luminance is the obtained brightness value, modify the image and maintain the original text arrangement; S37. Perform DC component recovery processing on the last character of the sequential text, set the DC component as the differential pre-prefix of the subsequent blocks for the last block of the continuously modified blocks, and save the result; S38. The modified image block undergoes discrete cosine transform, Zig-zag encoding, quantization, differential encoding of DC coefficients, and Huffman encoding through the encoding module to form new MCU data; S39. Once all data units of this image block have been encoded, write the new MCU to the output buffer.

2. The JPEG image partial modification method as described in claim 1, characterized in that, After receiving the JPEG header, it also includes: Receive image recognition information from the JPEG header, define the quantization table, basic image information, and define the Huffman table; Determine whether the JPEG encoding format is JFIF based on image recognition information, and determine the number of Y, Cb, and Cr components based on basic image information; After the header is parsed, the received JPEG header data is synchronously written to the output buffer.

3. The JPEG image partial modification method as described in claim 2, characterized in that, The step of directly writing the current MCU to the output buffer if the current MCU does not need to be modified includes: S41. If the current MCU does not need to be modified, start reading bit by bit from the starting address of the current MCU color component unit data stream until the encoded bit data string matches the encoding in the DC Huffman table of that component obtained in the header, and then query the DC Huffman table to obtain the corresponding DC component value. S42. Continue reading data bit by bit until the read code matches the code in the AC Huffman table for that component, then query the AC Huffman table to obtain the corresponding AC component value; S43. Repeat step S42 until the condition for the end of AC component data is met. There are two conditions for the end, and either one needs to be met: the value of the AC component read is zero, which means that all subsequent AC variables are zero; or all 63 AC components have been read. S44. Repeat S41 to S43 continuously until all MCUs in this image block have been read, and write the current MCU to the output buffer.

4. The JPEG image partial modification method as described in claim 1, characterized in that, The method for partially modifying a JPEG image also includes: S51. Read data sequentially from the output buffer, assemble the data into a message, and send it to the gateway; S52. Repeat step S51 continuously until the image data processing is complete and the data is assembled into complete JPEG image data.

5. A JPEG image partial modification device, characterized in that, The JPEG image partial modification device includes: The input module is used to receive data from the JPEG data stream, process it bit by bit, and read the data sequentially into a 256-byte input buffer; The determination module is used to receive the JPEG header, decode the smallest encoding unit (MCU) of the image sequentially according to the parameters read from the header, and determine whether the current MCU needs to be modified by comparing whether the coordinates of the current MCU are within the specified area of ​​the image to be modified. The first processing module is used for: S31. If the current MCU needs to be modified, start reading bit by bit from the beginning of the current MCU color component unit data stream. The reading steps are the same as S41 to S43. S32. Decode the differential DC component of each color component unit sequentially, using the formula... The current DC variable is obtained by correcting the actual DC variable of the previous color component unit, where It is a difference correction variable; S33. Use the quantization table decoded from the color component information field in the marker segment SOF to dequantize different color components. During the dequantization process, multiply each of the 64 values ​​of the 8×8 color component unit by the corresponding value in the same position in the quantization table. S34. Reverse Zig-zag encoding of each dequantized 8×8 color component unit according to its element number; S35. Perform an inverse discrete cosine transform to convert the values ​​of the color component unit matrix from the frequency domain to the spatiotemporal domain, and the size of the matrix in the spatiotemporal domain remains unchanged after the inverse discrete cosine transform. After decoding by the S36.MCU, a 16×16 pixel image in YUV format is obtained. A 16-bit dot matrix character library is selected to modify the data of a single image block. The character data is divided into two categories: half-width and full-width, occupying 16×8 and 16×16 pixels respectively. Based on the original text dot matrix, these are sequentially mapped to consecutive image blocks. Simultaneously, based on the brightness and complementary color principles of the block pixels, a formula is used... Determine the brightness value of the overlay text to increase the contrast between the overlay text and the background color, so that the text is clearly displayed, where luminance is the obtained brightness value, modify the image and maintain the original text arrangement; S37. Perform DC component recovery processing on the last character of the sequential text, set the DC component as the differential pre-prefix of the subsequent blocks for the last block of the continuously modified blocks, and save the result; S38. The modified image block undergoes discrete cosine transform, Zig-zag encoding, quantization, differential encoding of DC coefficients, and Huffman encoding through the encoding module to form new MCU data; S39. Once all data units of this image block have been encoded, write the new MCU into the output buffer; The second processing module is used to directly write the current MCU to the output buffer if the current MCU does not need to be modified.

6. The JPEG image partial modification apparatus as described in claim 5, characterized in that, The JPEG image partial modification device further includes a parsing module, used for: Receive image recognition information from the JPEG header, define the quantization table, basic image information, and define the Huffman table; Determine whether the JPEG encoding format is JFIF based on image recognition information, and determine the number of Y, Cb, and Cr components based on basic image information; After the header is parsed, the received JPEG header data is synchronously written to the output buffer.

7. The JPEG image partial modification apparatus as described in claim 6, characterized in that, The second processing module is used for: S41. If the current MCU does not need to be modified, start reading bit by bit from the starting address of the current MCU color component unit data stream until the encoded bit data string matches the encoding in the DC Huffman table of that component obtained in the header, and then query the DC Huffman table to obtain the corresponding DC component value. S42. Continue reading data bit by bit until the read code matches the code in the AC Huffman table for that component, then query the AC Huffman table to obtain the corresponding AC component value; S43. Repeat step S42 until the condition for the end of AC component data is met. There are two conditions for the end, and either one needs to be met: the value of the AC component read is zero, which means that all subsequent AC variables are zero; or all 63 AC components have been read. S44. Repeat S41 to S43 continuously until all MCUs in this image block have been read, and write the current MCU to the output buffer.

8. The JPEG image partial modification apparatus as described in claim 5, characterized in that, The JPEG image partial modification apparatus also includes an output module for: S51. Read data sequentially from the output buffer, assemble the data into a message, and send it to the gateway; S52. Repeat step S51 continuously until the image data processing is complete and the data is assembled into complete JPEG image data.

Images