Image transmission method, device, equipment and storage medium

Abstract

The present application provides an image transmission method, apparatus, device, and storage medium, relating to the field of data transmission. The method comprises: obtaining a first keyframe obtained by current encoding; determining the position information of a target keyframe and difference information between the first keyframe and the target keyframe based on the first keyframe and a preset first feature set; and transmitting the position information and difference information to a decoding device, so that the decoding device obtains the first keyframe based on the position information, the difference information, and a second feature set in the decoding device; the second feature set being the same as the first feature set. This method is applicable to image compression transmission and is used to achieve high-quality image transmission in weak network conditions.

Description

Technical Field

[0001] The present application relates to the field of data transmission, and in particular to an image transmission method, apparatus, device and storage medium. Background Art

[0002] Video sequences contain a range of redundant information, including spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. To minimize this redundant information and reduce the amount of data representing the video, video coding and decoding technologies have been proposed to reduce storage space and transmission bandwidth. Video coding and decoding technologies are also known as video compression technologies.

[0003] A system using video coding and decoding technology may generally include an encoder and a decoder. The code stream obtained after the video sequence is encoded by the encoder is transmitted to the decoder, and the decoder decodes the code stream to obtain the video sequence.

[0004] However, during the transmission of the code stream, there may be a weak network or unstable network, which may affect the code stream to be decoded received by the decoder. Summary of the Invention

[0005] Based on the above technical problems, the present application provides an image transmission method, apparatus, device and storage medium, which can determine the difference information between the key frame and the feature set by comparing the currently generated key frame (instant key frame) with the preset feature set (multiplexed key frame), so as to reduce the requirements of the transmission link and reduce the dependence on the network, thereby realizing image transmission in weak network or unstable network conditions.

[0006] In a first aspect, the present application provides an image transmission method, which includes: obtaining a first key frame obtained by current encoding; determining the position information of a target key frame and the difference information between the first key frame and the target key frame based on the first key frame and a first feature set including at least one historical key frame; the target key frame is a historical key frame in the first feature set, and the difference between the target key frame and the first key frame is less than a preset threshold; the position information is used to indicate the position of the target key frame in multiple historical key frames; the difference information is used to represent the difference between the first key frame and the target key frame; sending the position information and the difference information to a decoding device, so that the decoding device obtains the first key frame based on the position information, the difference information, and a second feature set in the decoding device; the second feature set is the same as the first feature set.

[0007] In one possible implementation, based on a first key frame and a first feature set including at least one historical key frame, the position information of a target key frame and the difference information between the first key frame and the target key frame are determined, including: comparing the first key frame with the historical key frames in the first feature set one by one in order from near to far in terms of the generation time of each historical key frame in the first feature set, and determining the first historical key frame whose difference is less than a preset threshold as the target key frame; and determining the position information and difference information based on the target key frame.

[0008] This section describes a specific implementation method for scenarios where images change regularly over time.

[0009] In another possible implementation, based on the first key frame and the preset first feature set, the position information of the target key frame and the difference information between the first key frame and the target key frame are determined, including: according to the order of the hit number of each historical key frame in the first feature set from most to least, the first key frame is compared with the historical key frames in the first feature set one by one, and the first historical key frame whose difference is less than a preset threshold is determined as the target key frame; the hit number is the number of times the historical key frame is selected as the target key frame; based on the target key frame, the position information and difference information are determined.

[0010] Here, a specific implementation method for a scene with a constant light source (such as an indoor scene) is described.

[0011] Optionally, the first feature set includes sorting the historical key frames in the candidate feature set in descending order of hit counts, and the sorting result is the top M historical key frames; the hit count is the number of times the historical key frame is selected as the target key frame; M is a positive integer.

[0012] Optionally, the method also includes: obtaining a draft set before obtaining the currently generated first key frame; the draft set includes one or more historical key frames; in the case where the stream information sent by the decoding device is not obtained, each historical key frame in the draft set is compared with the historical key frames in the first feature set; the stream information is used to indicate that the decoding device needs to perform decoding; the historical key frames in the draft set, whose differences with each historical key frame in the first feature set are greater than a preset threshold, are recorded as newly added historical key frames in the first feature set; and the newly added historical key frames are sent to the decoding device so that the decoding device records the newly added historical key frames in the second feature set.

[0013] Optionally, the draft set includes a first historical key frame; the method also includes: if there is a second historical key frame in the first feature set and the difference between the second historical key frame and the first historical key frame is less than a preset threshold, then the generation time of the second historical key frame in the first feature set is updated to the generation time of the first historical key frame.

[0014] The image transmission method provided by the present application, after obtaining the first key frame obtained by the current encoding, can compare the first key frame with the historical key frames in the preset first feature set, determine the position information of the target key frame that is smaller than the first key frame, and the difference information between the first key frame and the target key frame, and transmit the position information and difference information to the decoding device, so that the decoding device can restore the first key frame to complete the decoding based on the position information, difference information, and the second feature set preset in the decoding device. Compared with the original scheme of transmitting the first key frame, the image transmission method provided by the present application only needs to transmit the position information and difference information to realize the compressed transmission of the image (video) data after the feature sets are preset at the encoding end and the decoding end respectively. The amount of data to be transmitted is small, which reduces the requirements for the transmission link, and can realize lossless transmission of the image in the case of weak network or unstable network.

[0015] In a second aspect, the present application provides an image decoding method, which is applied to a decoding device, and the method includes: obtaining position information of a target key frame sent by an encoding device, and difference information between a first key frame and the target key frame; the position information and the difference information are determined by the encoding device based on the first key frame obtained by encoding and a first feature set including at least one historical key frame; the target key frame is a historical key frame in the first feature set, the difference between which and the first key frame is less than a preset threshold; the target key frame is determined from a second feature set in the decoding device according to the position information; the second feature set is the same as the first feature set; the first key frame is obtained according to the difference information and the target key frame, and decoding is performed according to the first key frame.

[0016] In a third aspect, the present application provides an image transmission device, which includes various functional modules used in the method described in the first aspect above.

[0017] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of the related method described in the first aspect to implement the method described in the first aspect.

[0018] In a fifth aspect, the present application provides an electronic device comprising a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect above.

[0019] In a sixth aspect, the present application provides a readable storage medium, which includes: software instructions; when the software instructions are executed in an electronic device, the electronic device implements the method described in the first aspect above.

[0020] The beneficial effects of the second to sixth aspects mentioned above can be referred to those described in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic diagram of the composition of an image transmission system provided in an embodiment of the present application;

[0023] Figure 2 A GOP diagram provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0025] Figure 4 A flowchart of an image transmission method provided in an embodiment of the present application;

[0026] Figure 5 Another flowchart of the image transmission method provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the composition of the image transmission device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] First, the terms involved in the embodiments of the present application are introduced.

[0029] 1. Video compression technology:

[0030] Video compression technology is also known as video codec technology. Video codec technology includes both video encoding and video decoding technologies. Video sequences contain a range of redundant information, including spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. To minimize redundant information in video sequences and reduce the amount of data representing the video, video coding technology can be used to encode video sequences, thereby reducing storage space and transmission bandwidth.

[0031] 2. I frame:

[0032] An intra-coded image frame, also known as a key frame, is an independently coded frame that carries all its own information. It can be decoded independently without referring to other image codes and can be understood as a static picture.

[0033] 3. P-frame:

[0034] The forward predictive coding image frame uses the I frame as a reference frame and encodes and saves the content different from the I frame. During decoding, the complete P frame image must be reconstructed based on the I frame.

[0035] 4. Other terms:

[0036] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0037] A system using video coding and decoding technology may generally include an encoder and a decoder. The code stream obtained after the video sequence is encoded by the encoder is transmitted to the decoder, and the decoder decodes the code stream to obtain the video sequence.

[0038] However, during the transmission of the code stream, there may be a weak network or an unstable network, which may affect the code stream to be decoded received by the decoder, for example, it may cause frame loss.

[0039] On this basis, an embodiment of the present application provides an image transmission method, which can compare the currently generated key frame with the preset feature set, transmit the difference information between the key frame and the feature set, so as to reduce the requirements of the transmission link, thereby realizing image transmission in weak network or unstable network conditions.

[0040] The following is an introduction with reference to the accompanying drawings.

[0041] Figure 1 This is a schematic diagram of the composition of the image transmission system provided in the embodiment of the present application. Figure 1 As shown, the image transmission system may include: an encoding device 100, a key frame processing device 200, and a decoding device 300. The encoding device 100, the key frame processing device 200, and the decoding device 300 may be connected via a wired network or a wireless network.

[0042] The encoding device 100 may be an electronic device with encoding function, such as a camera, an encoder, a hard disk recorder, a computer, or a server, etc. This embodiment of the present application does not limit this.

[0043] The encoding device 100 can be used to encode the acquired video data (or image data, such as the above-mentioned video sequence) according to a specific compression standard to obtain encoded video data (code stream).

[0044] Illustratively, the encoded video data may include one or more groups of pictures (GOPs), and each GOP may include one I frame and one or more P frames.

[0045] For example, Figure 2 This is a GOP diagram provided in the embodiment of the present application. Figure 2 As shown, taking the coded video data of a video sequence (segment) as an example, the coded video data of the video sequence may include a video sequence header, GOP1, GOP2, ..., GOPn, and a video sequence tail. Among them, GOP1 may include I frame ( Figure 2 The box with the letter "I" is shown as an example), several P frames ( Figure 2 ) is shown in the example with the letter "P".

[0046] In some embodiments, encoding device 100 can also be used to acquire image data. For example, encoding device 100 can be connected to an image acquisition device via a wired or wireless network, and encoding device 100 can receive image data sent by the image acquisition device. For another example, encoding device 100 can also integrate an image acquisition module and have image acquisition capabilities, such as an IPC (IP Camera) network camera.

[0047] In some embodiments, the encoding apparatus 100 may further be configured to send the encoded video data to the key frame processing apparatus 200 .

[0048] As described above, the encoding apparatus 100, the key frame processing apparatus 200, and the decoding apparatus 300 may be connected via a wired network or a wireless network. The wired network or the wireless network may include one or more media or devices capable of transmitting the encoded video data from the encoding apparatus 100 to the key frame processing apparatus 200.

[0049] In some embodiments, the wired or wireless network may include one or more communication media that enable the encoding device 100 to send the encoded video data directly to the key frame processing device 200 in real time. In this embodiment, the encoding device 100 can modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and send the modulated encoded video data to the key frame processing device 200. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. Optionally, the one or more communication media may form part of a packet-based network, which may be, for example, a local area network, a wide area network, or a global network (e.g., the Internet). Optionally, the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the encoding device 100 to the key frame processing device 200.

[0050] The key frame processing device 200 may be an electronic device with computing capabilities.

[0051] The key frame processing device 200 can be used to process the encoded video data obtained by encoding by the encoding device 100. The specific process can be referred to the image transmission method described below, and will not be repeated here.

[0052] In some embodiments, the key frame processing device 200 is further configured to send the processed encoded video data to the decoding device 300 .

[0053] The specific sending process can refer to the above-mentioned part where the encoding device 100 sends the encoded video to the key frame processing device 200, and will not be repeated here.

[0054] The decoding device 300 may be an electronic device with a decoding function, such as a decoder, a hard disk recorder, a computer, or a server.

[0055] The decoding device 300 is used to decode the acquired encoded video data according to a specific compression standard.

[0056] In some embodiments, the decoding device 300 may also be configured to receive the encoded video data sent by the key frame processing device 200 .

[0057] It should be noted that the above Figure 1 The encoding device 100, key frame processing device 200, and decoding device 300 are illustrated as independent devices. Alternatively, the key frame processing device 200 and the encoding device 100 may be combined into a single device. That is, the encoding device 100 or its corresponding functions and the key frame processing device 200 or its corresponding functions may be integrated into the same device. For example, an encoding device having a key frame processing function may be provided. This embodiment of the present application is not limited to this.

[0058] The execution subject of the image transmission method provided in the embodiment of the present application is the above-mentioned key frame processing device 200 or a device in which the key frame processing device 200 and the encoding device 100 are combined. The key frame processing device 200 or the device in which the key frame processing device 200 and the encoding device 100 are combined can be an electronic device with computing and processing functions, such as a camera, an encoder, a hard disk recorder, a computer or a server. Optionally, the device can also be a processor in the above-mentioned electronic device; or, the device can also be an application installed in the above-mentioned electronic device that provides an image transmission function; or, the device can also be a functional module in the above-mentioned electronic device for performing the image transmission function, etc. The embodiment of the present application does not limit this.

[0059] For simplicity of description, the following description will take the key frame processing device 200 or a device in which the key frame processing device 200 and the encoding device 100 are jointly provided as an example of the above-mentioned electronic device.

[0060] Figure 3 This is a schematic diagram of the composition of the electronic device provided in the embodiment of the present application. Figure 3 As shown, the electronic device may include:

[0061] A processor 10 , a memory 20 , a communication line 30 , a communication interface 40 , and an input / output interface 50 .

[0062] The processor 10 , the memory 20 , the communication interface 40 , and the input / output interface 50 may be connected via a communication line 30 .

[0063] The processor 10 is used to execute the instructions stored in the memory 20 to implement the image transmission method provided in the following embodiments of the present application. The processor 10 can be a central processing unit (CPU), a general-purpose processor network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller (MCU) / single chip microcomputer / single chip microcomputer, a programmable logic device (PLD) or any combination thereof. The processor 10 can also be any other device with processing functions, such as a circuit, a device or a software module, which is not limited in the embodiments of the present application. In one example, the processor 10 may include one or more CPUs, such as Figure 3 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 10, it may also include a processor 60 ( Figure 3 The dashed line is used as an example.

[0064] Memory 20 is used to store instructions. For example, the instruction can be a computer program. Optionally, the memory 20 can be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a CD-ROM or other optical disk storage, an optical disk storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, etc., and the embodiments of the present application are not limited to this.

[0065] It should be noted that the memory 20 may exist independently of the processor 10 or may be integrated with the processor 10. The memory 20 may be located inside the electronic device or outside the electronic device, which is not limited in the embodiment of the present application.

[0066] The communication line 30 is used to transmit information between the components included in the electronic device.

[0067] Communication interface 40 is used to communicate with other devices (e.g., encoding device 100 or decoding device 300) or other communication networks. Such other communication networks may be Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc. Communication interface 40 may be a module, circuit, transceiver, or any other device capable of communication.

[0068] The input / output interface 50 is used to implement human-computer interaction between a user and the electronic device, such as action interaction, text interaction, or voice interaction.

[0069] For example, the input / output interface 50 may be a keyboard, a mouse, etc. Action interaction or text interaction between a user and the electronic device may be achieved through the keyboard, the mouse, etc.

[0070] It should be noted that Figure 3 The structure shown in the figure does not constitute a limitation on the electronic device, except Figure 3 In addition to the components shown, the electronic device may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0071] The image transmission method provided in the embodiments of the present application is introduced below with reference to the accompanying drawings.

[0072] Figure 4 Schematic diagram of the flow of the image transmission method provided in the embodiment of the present application. Figure 4 As shown, the method includes S101 to S103.

[0073] S101: The electronic device obtains a first key frame obtained by current encoding.

[0074] The electronic device can obtain the first key frame obtained by the current encoding by referring to the above Figure 1 As described above, the key frame processing device 200 obtains the encoded video data. The electronic device can receive the first key frame obtained by encoding and sent by the encoding device 100.

[0075] S102: The electronic device determines position information of a target key frame and difference information between the first key frame and the target key frame based on the first key frame and a preset first feature set.

[0076] The first feature set includes at least one historical keyframe. The target keyframe is a historical keyframe in the first feature set whose difference with the first keyframe is less than a preset threshold. The position information indicates the position of the target keyframe in the first feature set. The difference information indicates the difference between the first keyframe and the target keyframe.

[0077] Optionally, the preset threshold value can be preset in the electronic device by a manager. For example, the preset threshold value can be 10%, 15%, or a specific pixel value. The embodiments of the present application do not limit the specific type and value of the preset threshold value. As described above, the electronic device can include an input / output interface, such as a mouse or keyboard. The manager can set the preset threshold value in the electronic device using the mouse or keyboard.

[0078] Optionally, the position information may be a unique identifier such as a frame number or an identity identifier of the target key frame.

[0079] In one possible implementation, for a scenario where images change regularly over time, the electronic device can compare the first key frame with the historical key frames in the first feature set one by one in the order of the generation time of each historical key frame in the first feature set from near to far, and determine the first historical key frame whose difference is less than a preset threshold as the target key frame, and then determine the position information and difference information based on the target key frame.

[0080] Optionally, the electronic device can sort multiple historical key frames in the first feature set in order from near to far in terms of generation time to obtain a first sorting result, and compare the first key frame with the historical key frames in the first sorting result one by one, and determine the first historical key frame whose difference is less than a preset threshold as the target key frame.

[0081] For example, the electronic device may maintain a timetable for the historical key frames in the first feature set to record the generation time of each historical key frame.

[0082] Illustratively, the schedule for the electronic device to maintain historical key frames of the first feature set may be as shown in Table 1 below.

[0083] Table 1

[0084] Generation time The frame number of the historical keyframe Number of hits 0 o'clock 0:00 2 59 … … 0:59 23 44 1 o'clock 1:00 44 105 … … 1:59 49 101 2 o'clock … … … 3 o'clock … … … … … … 11 p.m. 11:00 PM 56 67 … … … 11:59 PM 78 0

[0085] As shown in Table 1, the table may include a generation time item, a historical key frame frame number item, and a hit count item. The generation time item may include 0:00, ..., 0:59, 1:00, ..., 1:59, ..., 23:00, ..., and 23:59. The historical key frame frame number item may include frame numbers such as 2, ..., 23, ..., 44, ..., 49, ..., 56, ..., and 78. The hit count item may include 59, ..., 44, 105, ..., 101, ..., 67, and 0, etc. There is a corresponding relationship between time 0:00, frame number 2, and hit number 59; there is a corresponding relationship between time 0:59, frame number 23, and hit number 44; there is a corresponding relationship between time 1:00, frame number 49, and hit number 105; there is a corresponding relationship between time 1:59, frame number 49, and hit number 101; there is a corresponding relationship between time 23:00, frame number 56, and hit number 67; there is a corresponding relationship between time 23:59, frame number 78, and hit rate 0.

[0086] It should be noted that the hit count is the number of times the historical key frame is determined as the target key frame. For a historical key frame No. 78 generated at 23:59, the hit count of the historical key frame is 0, that is, it has never been determined as a target key frame. The electronic device can replace the historical key frame No. 78 with other historical key frames in the next cycle to ensure the validity of the first feature set. The period here is the period between receiving the flow information and not receiving the flow information. For details, please refer to the following process of updating the first feature set according to the draft set, which will not be repeated here.

[0087] It should be noted that the time intervals in Table 1 above are only examples. Assuming that the timetable precision is minute, that is, a historical key frame is obtained every minute, the administrator can also select a different precision based on the device storage situation, such as 1 hour, 30 minutes, 10 minutes, or 5 minutes. This embodiment of the present application is not limited to this.

[0088] Optionally, the electronic device may use a differential method to compare the first key frame with the historical key frames. In this case, for the comparison between the first key frame and any one of the historical key frames in the first feature set, the electronic device compares the first key frame with the multiple historical key frames in the first feature set one by one in the order of their generation times from near to far. This may specifically include: the electronic device subtracts the pixel value of each pixel point in the first key frame from the pixel value of each pixel point in the historical key frame to obtain a differential image; and obtains a comparison result based on the differential image.

[0089] For example, the electronic device may use the sum of the pixel values ​​of the pixels in the differential image as the comparison result, or may use the quotient of the sum of the pixel values ​​of the pixels in the differential image and the sum of the pixel values ​​of the pixels in the first key frame as the comparison result. This embodiment of the present application is not limited to this.

[0090] Optionally, as described above, the electronic device may use a differential method to compare the first key frame and the historical key frame. In this case, the difference information between the first key frame and the target key frame may be a differential image obtained by subtracting the pixel value of each pixel point in the first key frame from the pixel value of each pixel point in the target key frame.

[0091] In another possible implementation, for a scenario with a constant light source (such as an indoor scene), the electronic device can compare the first key frame with the historical key frames in the first feature set one by one in order from most to least hit counts of each historical key frame in the first feature set, and determine the first historical key frame whose difference is less than a preset threshold as the target key frame, and then determine the position information and difference information based on the target key frame.

[0092] Optionally, the electronic device can sort multiple historical key frames in the first feature set in order from most to least number of hits to obtain a second sorting result, and compare the first key frame with the historical key frames in the second sorting result one by one, and determine the target key frame as the first historical key frame whose difference is less than a preset threshold.

[0093] Optionally, in the above-mentioned scenario where the light source is constant, the image does not change much over time. In order to save storage space, the size of the feature set can also be limited. In this case, before the above-mentioned S101, the method can also include: the electronic device obtains a candidate feature set, sorts the historical key frames in the candidate feature set in order from most to least according to the number of hits, and determines the historical key frames with the top M sorting results as the first feature set.

[0094] Among them, the candidate feature set can be the first feature set obtained by the first screening from the draft set described below. The candidate feature set can also include multiple sets. The specific screening process can refer to the draft set described below and will not be repeated here. M is a positive integer. M can be preset in the electronic device by the administrator. For example, M can be set to 10, 15, or 20. The embodiment of the present application does not limit the specific value of M.

[0095] Based on the above understanding, the first feature set can be understood as including the historical key frames in the candidate feature set sorted in descending order of the number of hits, and the sorting result is the top M historical key frames.

[0096] S103: The electronic device sends the position information and the difference information to the decoding device, so that the decoding device obtains the first key frame according to the position information, the difference information, and the second feature set in the decoding device.

[0097] The second feature set in the decoding device is the same as the first feature set in the electronic device.

[0098] For example, the electronic device may obtain a first feature set and send it to the decoding device, and the decoding device may obtain the first feature set sent by the electronic device and store it as a second feature set.

[0099] In one possible implementation, after acquiring the position information and difference information sent by the electronic device, the decoding device may determine the target key frame from the second feature set according to the position information, and obtain the first key frame according to the target key frame and the difference information.

[0100] Optionally, as described above, the electronic device may use the differential image obtained by subtracting the pixel value of each pixel point in the first key frame from the pixel value of each pixel point in the target key frame as the difference information. In this case, after the decoding device determines the target key frame from the second feature set based on the position information, it may add the pixel value of each pixel point in the target key frame to the pixel value of each pixel point in the differential image to obtain the first key frame.

[0101] The image transmission method provided in the embodiment of the present application can, after obtaining the first key frame obtained by the current encoding, compare the first key frame with the historical key frames in the preset first feature set, determine the position information of the target key frame that is less different from the first key frame, and the difference information between the first key frame and the target key frame, and transmit the position information and difference information to the decoding device, so that the decoding device can restore the first key frame to complete the decoding based on the position information, difference information, and the second feature set preset in the decoding device. Compared with the original scheme of transmitting the first key frame, the image transmission method provided in the embodiment of the present application only needs to transmit the position information and difference information to realize the compressed transmission of the image (video) data after the feature sets are preset at the encoding end and the decoding end respectively. The amount of data to be transmitted is small, which reduces the requirements for the transmission link, and can realize lossless transmission of the image in the case of weak network or unstable network.

[0102] Optionally, when the target key frame does not exist in the first feature set, the electronic device may send the first key frame to the decoding device so that the decoding device performs decoding according to the first key frame. The specific process can be referred to in the related art and will not be repeated here.

[0103] In some possible embodiments, before the above S101, the electronic device may further update the first feature set, which may specifically include the following steps:

[0104] Step 1: The electronic device obtains the draft set.

[0105] The draft set may also include multiple historical key frames.

[0106] Step 2: When the electronic device does not obtain the stream acquisition information sent by the decoding device, it compares each historical key frame in the draft set with the historical key frames in the first feature set.

[0107] The process of comparing each historical key frame in the draft set with the historical key frames in the first feature set can refer to the process of comparing the first key frame with the historical key frames in the first feature set, which will not be repeated here.

[0108] For example, the decoding device can be a playback device with a decoding function, which can receive the user's playback instructions and, in response to the playback instructions, send stream information to the encoding device or electronic device (key frame processing device), obtain the encoded video data, decode the encoded video data to obtain a video sequence and play it.

[0109] Optionally, when the electronic device obtains the stream acquisition information sent by the decoding device, it can transmit the encoded video data according to the above steps S101 to S103, which will not be repeated here.

[0110] Step 3: The historical key frames in the draft set whose differences with each historical key frame in the first feature set are greater than a preset threshold are recorded as newly added historical key frames in the first feature set.

[0111] The preset threshold value may refer to the above-mentioned S102 and will not be described in detail here.

[0112] Optionally, taking the first historical key frame included in the draft set as an example, the method may also include: if there is a second historical key frame in the first feature set and the difference between the second historical key frame and the first historical key frame is less than a preset threshold, the electronic device updates the generation time of the second historical key frame in the first feature set to the generation time of the first historical key frame.

[0113] Optionally, after the electronic device updates the generation time of the second historical key frame in the first feature set to the generation time of the first historical key frame, it can also send update information to the decoding device so that the decoding device updates the generation time of the second historical key frame in the second feature set to the generation time of the first historical key frame according to the update information.

[0114] Based on the above understanding, Figure 5 Another flow chart of the image transmission method provided in the embodiment of the present application. Figure 5 As shown, the method can be applied to an image transmission system, which may include an encoding device, a key frame processing device, and a decoding device. The method mainly includes a feature set generation mode (idle time) and an operating mode (busy time). The feature set generation mode is the case where the stream acquisition information sent by the decoding device is not obtained, and the operating mode is the case where the stream acquisition information sent by the decoding device is obtained.

[0115] In the feature set generation mode, the following steps are mainly included:

[0116] Step 1: The encoding device generates a key frame.

[0117] Step 2: The encoding device records the key frame and generates a unique ID, and records it in a cache table, which is recorded as a draft set.

[0118] Step 3: The key frame processing device compares the differences between the feature set image and the draft set image.

[0119] If the difference is less than the threshold, execute step 4 and then step 6; if the difference is greater than the threshold, execute step 5 and then step 6.

[0120] Step 4: The key frame processing device updates the timetable corresponding to the feature set picture.

[0121] Step 5: The key frame processing device moves the pictures in the draft set to the feature set.

[0122] Step 6: The decoding device synchronously updates the feature set.

[0123] Steps 1-6 in the feature set generation mode may refer to those described in the above possible embodiment and will not be described in detail here.

[0124] In working mode, the following steps are mainly included:

[0125] Step 1: The encoding device generates an instant key frame.

[0126] Step 2: The encoding device compares the difference between the feature set image and the instant key frame.

[0127] If the difference is less than the threshold, execute step 3 and then step 5; if the difference is greater than the threshold, execute step 4 and then step 6.

[0128] Step 3: The encoding device sends the ID of the multiplexed key frame and the difference frame information to the decoding device.

[0129] Step 4: The encoding device sends the instant key frame to the decoding device, and records the instant key frame into the draft set.

[0130] Step 5: The decoding device obtains the key frame from the feature set according to the ID of the multiplexed key frame, and obtains the video data according to the difference frame information.

[0131] Step 6: The decoding device calculates and obtains video data based on the instant key frame (original frame decoding process).

[0132] Optionally, if the difference in step 2 is equal to the threshold, step 5 may be performed after step 3, or step 6 may be performed after step 4. This embodiment of the present application does not limit this.

[0133] Steps 1-6 in the working mode can refer to the above-mentioned S101 to S103 and will not be repeated here.

[0134] Based on the understanding of step 6 above, an embodiment of the present application further provides an image decoding method, which is applied to a decoding device and includes the following steps:

[0135] Step 1: The decoding device obtains the position information of the target key frame sent by the encoding device, and the difference information between the first key frame and the target key frame.

[0136] Among them, the position information and difference information are determined by the encoding device based on the first key frame obtained by encoding and the first feature set including at least one historical key frame; the target key frame is a historical key frame in the first feature set, the difference between which and the first key frame is less than a preset threshold.

[0137] Step 2: The decoding device determines the target key frame from the second feature set in the decoding device according to the position information.

[0138] The second feature set is the same as the first feature set.

[0139] Step 3: The decoding device obtains a first key frame according to the difference information and the target key frame, and performs decoding according to the first key frame.

[0140] Optionally, as described above, the difference information may be a differential image, and the decoding apparatus may add the differential image and the target key frame according to the corresponding pixel value of each pixel to obtain the first key frame.

[0141] The process of decoding by the decoding device according to the first key frame can be referred to the description in the related art and will not be repeated here.

[0142] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed herein, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0143] In an exemplary embodiment, the present application also provides an image transmission device, Figure 6 This is a schematic diagram of the composition of the image transmission device provided in the embodiment of the present application. Figure 6 As shown, the device includes: an acquisition module 601, a processing module 602, and a sending module 603.

[0144] The acquisition module 601 is used to acquire the first key frame obtained by current encoding.

[0145] Processing module 602 is used to determine the position information of the target key frame and the difference information between the first key frame and the target key frame based on the first key frame and the first feature set including at least one historical key frame; the target key frame is a historical key frame in the first feature set, the difference between which the target key frame and the first key frame is less than a preset threshold.

[0146] The sending module 603 is used to send the position information and difference information to the decoding device, so that the decoding device obtains the first key frame according to the position information, difference information and the second feature set in the decoding device; the second feature set is the same as the first feature set.

[0147] In some possible embodiments, the processing module 602 is specifically used to compare the first key frame with the historical key frames in the first feature set one by one in the order of the generation time of each historical key frame from near to far, and determine the first historical key frame whose difference is less than a preset threshold as the target key frame; based on the target key frame, determine the position information and difference information.

[0148] In some other possible embodiments, the processing module 602 is specifically used to compare the first key frame with the historical key frames in the first feature set one by one in descending order of the hit count of each historical key frame in the first feature set, and determine the first historical key frame whose difference is less than a preset threshold as the target key frame; the hit count is the number of times the historical key frame is selected as the target key frame; based on the target key frame, determine the position information and difference information.

[0149] In some other possible embodiments, the first feature set includes sorting the historical key frames in the candidate feature set in descending order according to the number of hits, and the sorting result is the top M historical key frames; the number of hits is the number of times the historical key frame is selected as the target key frame; M is a positive integer.

[0150] In some other possible embodiments, the acquisition module 601 is further used to obtain a draft set before obtaining the currently generated first key frame; the draft set includes one or more historical key frames; the processing module 602 is further used to compare each historical key frame in the draft set with the historical key frames in the first feature set when the stream information sent by the decoding device is not obtained; the stream information is used to indicate that the decoding device needs to perform decoding; the historical key frames in the draft set whose differences with each historical key frame in the first feature set are greater than a preset threshold are entered into the first feature set as newly added historical key frames; the processing module 602 is further used to send the newly added historical key frames to the decoding device, so that the decoding device enters the newly added historical key frames into the second feature set.

[0151] In some other possible embodiments, the draft set includes a first historical key frame; the processing module 602 is further used to update the generation time of the second historical key frame in the first feature set to the generation time of the first historical key frame if there is a second historical key frame in the first feature set whose difference with the first historical key frame is less than a preset threshold.

[0152] It should be noted that Figure 6The module division described in the preceding text is illustrative and represents only one logical functional division. In actual implementation, other divisions may be employed. For example, two or more functions may be integrated into a single processing module. This is not a limitation of the present application. The integrated modules described above may be implemented in either hardware or software functional modules.

[0153] In an exemplary embodiment, the present application also provides a readable storage medium, including: software instructions, which, when executed on an electronic device, enable the electronic device to execute any one of the methods provided in the above embodiments.

[0154] In an exemplary embodiment, the present application also provides a computer program product including computer-executable instructions, which, when executed on an electronic device, enables the electronic device to execute any one of the methods provided in the above embodiments.

[0155] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer-executable instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), etc.

[0156] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0157] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

[0158] The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image transmission method, characterized in that: The method comprises: Get the first key frame obtained by the current encoding; In a scenario where an image changes regularly over time, the first key frame is compared with the historical key frames in the first feature set one by one in order of the generation time of each historical key frame from near to far, and the first historical key frame whose difference is less than a preset threshold is determined as a target key frame; based on the target key frame, position information of the target key frame and difference information between the first key frame and the target key frame are determined; In a scene with a constant light source, the first key frame is sequentially compared with the historical key frames in the first feature set in descending order of the hit count of each historical key frame in the first feature set, and the first historical key frame whose difference is less than the preset threshold is determined as the target key frame; based on the target key frame, the position information and the difference information are determined; the first feature set includes the historical key frames in the candidate feature set sorted in descending order of the hit count, and the sorting result is the top M historical key frames; the hit count is the number of times the historical key frame is selected as the target key frame; M is a positive integer; The position information and the difference information are sent to a decoding device so that the decoding device obtains the first key frame according to the position information, the difference information, and a second feature set in the decoding device; the second feature set is the same as the first feature set.

2. The method according to claim 1, characterized in that The method further comprises: Before obtaining the currently generated first key frame, obtaining a draft set; the draft set includes one or more historical key frames; In the case where the stream acquisition information sent by the decoding device is not obtained, each historical key frame in the draft set is compared with the historical key frames in the first feature set; the stream acquisition information is used to indicate that the decoding device needs to perform decoding; Recording the historical key frames in the draft set whose differences with each historical key frame in the first feature set are greater than the preset threshold as newly added historical key frames into the first feature set; The newly added historical key frame is sent to the decoding device, so that the decoding device records the newly added historical key frame into the second feature set.

3. The method according to claim 2, characterized in that The draft set includes a first historical key frame; the method further includes: If there is a second historical key frame in the first feature set whose difference with the first historical key frame is less than the preset threshold, the generation time of the second historical key frame in the first feature set is updated to the generation time of the first historical key frame.

4. An image decoding method, characterized in that: The method is applied to a decoding device, and includes: Obtaining position information of a target key frame sent by an encoding device, and difference information between a first key frame and the target key frame; the position information and the difference information are determined by the encoding device according to the method of any one of claims 1 to 3 based on the encoded first key frame and a first feature set including at least one historical key frame; the target key frame is a historical key frame in the first feature set whose difference with the first key frame is less than a preset threshold; determining the target key frame from a second feature set in the decoding device according to the position information; the second feature set is the same as the first feature set; The first key frame is obtained according to the difference information and the target key frame, and decoding is performed according to the first key frame.

5. An image transmission device, characterized in that: The device includes: an acquisition module, a processing module, and a sending module; The acquisition module is used to acquire the first key frame obtained by the current encoding; The processing module is used to, in a scenario where the image changes regularly over time, sequentially compare the first key frame with the historical key frames in the first feature set in the order of the generation time of each historical key frame from near to far, and determine the first historical key frame with a difference less than a preset threshold as the target key frame; based on the target key frame, determine the position information of the target key frame and the difference information between the first key frame and the target key frame; in a scenario where the light source is constant, sequentially compare the first key frame with the historical key frames in the first feature set in the order of the hit number of each historical key frame in the first feature set from most to least, and determine the first historical key frame with a difference less than the preset threshold as the target key frame; based on the target key frame, determine the position information and the difference information; the first feature set includes the historical key frames in the candidate feature set sorted in the order of the hit number from most to least, and the sorting result is the top M historical key frames; the hit number is the number of times the historical key frame is selected as the target key frame; M is a positive integer; The sending module is used to send the position information and the difference information to the decoding device, so that the decoding device obtains the first key frame according to the position information, the difference information and the second feature set in the decoding device; the second feature set is the same as the first feature set.

6. The device according to claim 5, characterized in that The acquisition module is further configured to acquire a draft set before acquiring the currently generated first key frame; the draft set includes one or more historical key frames; the processing module is further configured to compare each historical key frame in the draft set with the historical key frames in the first feature set when the stream acquisition information sent by the decoding device is not acquired; the stream acquisition information is used to indicate to the decoding device that decoding is required; and the historical key frames in the draft set whose differences with each historical key frame in the first feature set are greater than the preset threshold are entered into the first feature set as newly added historical key frames; The processing module is further configured to send the newly added historical key frame to the decoding device, so that the decoding device records the newly added historical key frame into the second feature set; and / or, The draft set includes a first historical key frame; the processing module is also used to update the generation time of the second historical key frame in the first feature set to the generation time of the first historical key frame if there is a second historical key frame in the first feature set whose difference with the first historical key frame is less than the preset threshold.

7. An electronic device, characterized in that: The electronic device includes: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 4.

8. A readable storage medium, characterized in that: The readable storage medium includes: software instructions; When the software instructions are executed in an electronic device, the electronic device is enabled to implement the method according to any one of claims 1 to 4.

Images