Method for optimizing image decoding process by using context information

Abstract

The invention relates to the technical field of image processing, and particularly discloses a method for optimizing an image decoding process by using context information. The method comprises the following steps: acquiring an image sequence and environment state data in real time, and constructing a calibrated image sequence and a context information sequence; analyzing a dynamic change mode of the calibrated image sequence by using a deep learning model, identifying a key region and a key frame sequence, and generating a final decoding strategy; according to the final decoding strategy, high-resolution decoding is carried out on the key area and the key frame, and low-resolution decoding is carried out on the non-key area and the non-key frame; dynamically adjusting a decoding process in combination with a network bandwidth condition, integrating a decoding result, and inputting the integrated decoding result into a seeker display system; and further optimizing the decoding strategy based on the real-time feedback data. According to the method, the decoding quality of the key area and the frame is effectively improved, the utilization of bandwidth resources is optimized, and the decoding efficiency and the image quality are remarkably improved in a scene with limited bandwidth and high real-time requirement.

Description

Technical Field

[0001] The present invention relates to the fields of new generation information technology and image processing technology, and in particular to a method for optimizing an image decoding process by utilizing context information. Background Art

[0002] With the development of image transmission and decoding technology, real-time image processing is increasingly used in scenarios such as seekers, monitoring equipment, and drones. However, in environments with limited bandwidth and high real-time requirements, there is often a contradiction between the efficiency and quality of image decoding. Traditional image decoding methods are usually unable to balance high-quality decoding of key image areas and overall decoding efficiency under limited bandwidth, resulting in the loss of details in dynamically changing areas, affecting the accuracy of target recognition and tracking.

[0003] Context information refers to information extracted from the temporal dynamic characteristics of image sequences, target motion states, and environmental states, which can provide important support for optimizing image decoding. However, most current decoding methods fail to fully utilize context information to efficiently decode dynamically changing areas of images. In particular, in scenarios where key areas and non-key areas require differentiated decoding, there is a lack of an optimized decoding strategy that comprehensively considers network bandwidth, real-time performance, and the importance of image areas, making it difficult to meet the application requirements of high precision and high real-time performance.

[0004] Therefore, there is an urgent need for an image decoding method that utilizes contextual information to dynamically analyze the target motion state and environmental characteristics, optimize the decoding strategy, improve the decoding quality of key areas, and reduce the decoding resource consumption of non-key areas, so as to achieve efficient image decoding and transmission under bandwidth-constrained conditions. Summary of the invention

[0005] The present invention provides a method for optimizing an image decoding process by utilizing context information, so as to solve the problem of how to optimize the image decoding process by utilizing context information in an environment with limited bandwidth and high real-time requirements, so as to improve the image quality and decoding efficiency of key areas.

[0006] In order to solve the above technical problems, the present invention provides a method for optimizing an image decoding process by using context information, comprising: Acquire an image sequence and environmental status data in real time, wherein the image sequence includes more than one continuous frame image; preprocess the image sequence and calibrate the moving target to obtain the motion state of the moving target area, and generate a calibrated image sequence; construct a context information sequence according to the calibrated image sequence and environmental status data; Based on the context information sequence, a deep learning model is used to analyze the dynamic change pattern of the calibrated image sequence, identify key areas, non-key areas, key frame sequences and non-key frame sequences, and generate a final decoding strategy in combination with the motion state of the target area; According to the final decoding strategy, the key area and key frame sequence are decoded at high resolution to obtain a decoded high-resolution image area, and the non-key area and non-key frame sequence are decoded at low resolution to obtain a decoded low-resolution image area; and the image decoding process of the calibrated image sequence is dynamically adjusted according to the network bandwidth status; The decoded high-resolution image area and low-resolution image area are integrated to obtain an integrated image sequence; the integrated image sequence is input into the seeker display system to obtain real-time feedback data; the final decoding strategy is adjusted based on the real-time feedback data to further optimize the image decoding process.

[0007] Furthermore, the step of acquiring the image sequence and environmental status data in real time also includes: Continuous frame images are acquired from an image acquisition device in real time and form an image sequence; environmental status data associated with the image sequence is acquired from an environmental monitoring device, the environmental status data including: lighting conditions, weather conditions, relative movement speed and / or temperature of the target.

[0008] Furthermore, the step of generating a calibrated image sequence further includes: The image sequence is denoised using an image denoising algorithm to obtain a preprocessed image sequence; the moving target in each frame of the preprocessed image sequence is located and calibrated using a target detection algorithm to obtain the motion state of the moving target area and generate a calibrated image sequence.

[0009] Furthermore, the step of constructing a context information sequence specifically includes: Context information is extracted according to the calibrated image sequence and environmental status data, and a context information sequence is constructed in combination with the time sequence; the context information sequence includes the time series features, spatial features and environmental status data of the calibrated image sequence.

[0010] Furthermore, the step of analyzing the dynamic change pattern in the calibrated image sequence using a deep learning model based on the context information sequence specifically includes: Inputting the context information sequence into a deep learning model, wherein the deep learning model is an LSTM network and / or a CNN network, wherein the LSTM network is used to extract time series features and learn the change trend of the calibrated image sequence in the time dimension, and the CNN network is used to extract spatial features and detect moving targets and key areas in the calibrated image sequence; The area with significant dynamic changes in each frame of the identified calibrated image sequence is regarded as the key area, and the area other than the key area is regarded as the non-key area; The frame sequences with significant dynamic changes are identified as key frame sequences, and the frame sequences other than the key frame sequences are identified as non-key frame sequences.

[0011] Furthermore, the step of generating the final decoding strategy includes: Based on the key area, the non-key area, the key frame sequence and the non-key frame sequence, and in combination with the motion state of the target area, a preliminary decoding strategy is generated; According to the real-time network bandwidth status and image transmission requirements, the preliminary decoding strategy is adjusted to generate a final decoding strategy.

[0012] Furthermore, the step of performing high-resolution decoding on the key area and key frame sequence according to the final decoding strategy to obtain a decoded high-resolution image area further includes: The key area and key frame are decoded with high resolution using a standard image compression algorithm to obtain a high resolution image area; Compressed sensing technology is used to perform low-resolution decoding on non-key areas and non-key frames to obtain low-resolution image areas.

[0013] Furthermore, the adjusting the final decoding strategy based on the real-time feedback data also includes: The final decoding strategy is adjusted based on the real-time feedback data of the seeker display system, the real-time feedback data including the network bandwidth status and / or the image quality score, to obtain a new decoding strategy.

[0014] Furthermore, the step of adjusting the final decoding strategy based on the real-time feedback data further includes: Continuously monitor the image decoding process to obtain monitoring data including image delay and bandwidth usage; adjust new decoding strategies based on the monitoring data.

[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) Optimized decoding strategy based on context information: The present invention combines image sequences and environmental state data, extracts the motion state characteristics of moving targets, and generates context information sequences, thereby accurately identifying key areas and key frame sequences in the image. Compared with traditional image decoding methods, the present invention uses this adaptive decoding strategy based on context information to effectively reduce unnecessary data transmission under limited bandwidth conditions, prioritize high-quality decoding of key areas, thereby improving decoding efficiency and saving bandwidth.

[0016] (2) Deep learning analysis of dynamic change patterns: By using deep learning models (LSTM and CNN) to analyze dynamic change patterns in image sequences, it is possible to identify areas with significant changes in the image and formulate high-resolution decoding strategies for these areas, while performing low-resolution processing on static or less changing areas. This innovation effectively improves image detail retention while optimizing the decoding process, avoiding over-decoding and bandwidth waste in traditional methods.

[0017] (3) Real-time feedback and decoding strategy adjustment: By inputting real-time feedback data (such as image clarity, delay, etc.) into the system and adjusting the decoding strategy in real time, the present invention can optimize image quality and real-time performance according to the dynamic changes in network bandwidth and transmission requirements. This feedback mechanism ensures that image decoding is always in the best state under the ever-changing network environment, ensuring image quality in high-real-time applications such as the seeker display system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a method for optimizing an image decoding process by utilizing context information is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] Example 1: Reference Figure 1 , is a flow chart of a method for optimizing an image decoding process by using context information provided by an embodiment of the present invention, and the flow may at least include steps S100-S400: S100, acquiring an image sequence and environmental status data in real time, the image sequence including more than one continuous frame image; performing preprocessing and motion target calibration on the image sequence to obtain the motion state of the motion target area, and generating a calibrated image sequence; constructing a context information sequence according to the calibrated image sequence and environmental status data.

[0022] S200. Based on the context information sequence, a deep learning model is used to analyze the dynamic change pattern of the calibrated image sequence, identify key areas, non-key areas, key frame sequences and non-key frame sequences, and generate a final decoding strategy in combination with the motion state of the target area.

[0023] S300. According to the final decoding strategy, the key area and the key frame sequence are decoded at high resolution to obtain a decoded high-resolution image area, and the non-key area and the non-key frame sequence are decoded at low resolution to obtain a decoded low-resolution image area; and according to the network bandwidth conditions, the image decoding process of the calibrated image sequence is dynamically adjusted.

[0024] S400, integrating the decoded high-resolution image area and the low-resolution image area to obtain an integrated image sequence; inputting the integrated image sequence into the seeker display system to obtain real-time feedback data; adjusting the final decoding strategy based on the real-time feedback data to further optimize the image decoding process.

[0025] Step S100 at least includes steps S110-S130: S110: Acquire image sequences and environmental status data in real time.

[0026] First, real-time continuous frame images are acquired from image acquisition devices such as optical cameras and composed into image sequence I: ; in, Indicates Frame image. Each frame image It consists of a group of pixels, representing the image data captured in real time.

[0027] Further, obtaining the environmental status data of the image sequence ,in Indicated in The environmental status data at the moment. The environmental status data includes external factors such as lighting conditions, weather conditions, relative motion speed of the target and / or temperature, which affect the quality of the image and the recognition of the target. Environmental status data It can be obtained through environmental monitoring equipment or sensors.

[0028] Furthermore, combining the real-time image sequence I and the environmental status data , construct a multimodal dataset containing time series and spatial information: ; in, It is the environmental state data at time t, including external parameters such as external lighting conditions, temperature, weather conditions, relative motion speed of the target, etc. D is the constructed multimodal dataset, including the image data at each moment. and the corresponding environmental status data .

[0029] The multimodal dataset contains the image frame and the corresponding environmental status data at each moment, providing necessary input for subsequent image analysis and context information extraction.

[0030] S120: preprocessing the image sequence to obtain a preprocessed image sequence, calibrating the moving target according to the preprocessed image sequence to obtain a target area, and generating a calibrated image sequence.

[0031] First, for each frame of image acquired Perform noise removal processing. The noise source comes from the sensor itself, the image compression algorithm or the external environment interference. Use the image denoising algorithm (Gaussian filter or median filter) to smooth the image and remove the random noise in the image to obtain the preprocessed image, thereby obtaining the preprocessed image sequence.

[0032] The target detection algorithm (YOLO, RCNN, etc.) is used to locate and calibrate the moving target in each frame of the preprocessed image sequence to obtain the motion state of the moving target area. Specifically, the target detection network is used to output the position, shape and dynamic characteristics of the moving target in each frame of the image to obtain the motion state of the target area: ; in, Indicates The motion state of the target area of ​​the frame image; Indicates Frame image The bounding box parameters of a moving object.

[0033] Furthermore, each frame of the image is processed by preprocessing and motion target calibration. , generate the calibrated image sequence: ; in, Indicates the calibrated Frame image; It is a calibrated image sequence, including a set of images after denoising and target calibration.

[0034] The calibrated image sequence is used as a new image sequence input to prepare for subsequent context information extraction.

[0035] S130: extracting context information from the calibrated image sequence and the environmental status data, and constructing a context information sequence in combination with the time sequence.

[0036] First, according to the calibrated image sequence and the corresponding environmental status data , extract context information. Specifically, extract the dynamic change pattern and motion state of the moving target area in each frame of the calibrated image sequence, and combine it with the environmental state data to generate context information.

[0037] The context information includes the time series characteristics of the calibrated image sequence, the spatial position of the target, the motion state (speed, acceleration), and the environmental state data (lighting, temperature, etc.).

[0038] Furthermore, the extracted context information is arranged in chronological order to obtain a context information sequence: ; in, Indicates time Contextual information at the moment; is the first Frame image, The context information sequence C provides the dynamic changes of the calibrated image sequence and the impact of environmental changes, providing support for the subsequent decoding strategy generation.

[0039] Step S200 at least includes steps S210-S230: S210: Based on the context information sequence, a deep learning model is used to analyze the dynamic change pattern of the calibrated image sequence to identify key areas, non-key areas, key frame sequences, and non-key frame sequences in the image.

[0040] First, the context information sequence generated by the S130 module is input into the deep learning model. Contains the spatial and temporal features in the image.

[0041] The deep learning model includes an LSTM network and / or a CNN network, which is used to analyze the dynamic change pattern in the calibrated image sequence. Among them, the LSTM network focuses on the modeling of time series data and can capture the dynamic changes across frames in the image sequence; while the CNN network can extract the spatial features in each frame of the image to detect moving targets and key areas in the image.

[0042] Among them, there are LSTM network (Long Short-Term Memory) and CNN network (Convolutional Neural Network).

[0043] The deep learning model is used to analyze the dynamic change patterns of the calibrated image sequence and identify key areas, non-key areas, key frame sequences and non-key frame sequences. Specifically: The identified areas with significant dynamic changes in the image are regarded as key areas, and the areas other than the key areas are non-key areas; the identified frame sequences with significant dynamic changes are regarded as key frame sequences, and the frame sequences other than the key frame sequences are regarded as non-key frame sequences. Specifically: ; A key area indicates an area in the image where significant changes have occurred (such as target movement or environmental changes). The key area is the part that needs to be focused on in the subsequent decoding strategy; ; is a key frame sequence, represents the jth key frame, where j is the index in the frame sequence. The key frame sequence will be decoded preferentially in the subsequent steps.

[0044] It should be noted that, in this embodiment, whether the dynamic change is significant can be quantified by measurement methods such as feature difference, motion vector, optical flow, etc. The selection of a specific method depends on the application scenario and actual needs.

[0045] S220: Generate a preliminary decoding strategy based on the motion state of the target area.

[0046] Combined with the key areas identified in S210 and keyframe sequences , and the motion state of the motion target area , generate a preliminary decoding strategy . Generate a preliminary decoding strategy according to the following rules : ; in, Indicates that key regions and key frame sequences are decoded with high priority. Indicates that non-key areas and non-key frames are decoded with low priority.

[0047] S230: adjusting the preliminary decoding strategy according to the real-time network bandwidth status and image transmission requirements to generate a final decoding strategy.

[0048] First, generate a preliminary decoding strategy After that, further combined with the real-time network bandwidth and image transmission requirements , where network bandwidth Indicates the network transmission rate, while image transmission requires Represents the bandwidth requirements of different image regions during transmission, dynamically adjusts the priority in the preliminary decoding strategy, thereby optimizing the preliminary decoding strategy and generating the final decoding strategy . It can be adjusted by the following formula: ; in, is the key area, indicating the preliminary decoding strategy A single region from the collection of image regions in . The priority of a traffic depends on its dynamic change level and transmission requirements. Indicates key areas Belongs to the preliminary decoding strategy set An image region in . To find the maximum function, it means to select the strategy that can maximize the priority from all possible decoding strategies; is an optimization function used to evaluate the and image transmission requirements Next, key areas By optimizing this function, the efficiency and adaptability of the decoding strategy can be ensured.

[0049] It should be noted that the optimization goal is to ensure that the transmission of important areas (i.e., key areas in this embodiment) and key frames can be guaranteed first, and in the case of limited bandwidth, low bandwidth or low resolution transmission is performed for unimportant areas (i.e., non-key areas in this embodiment) or non-key frames. and image transmission requirements , generating the final decoding strategy ,This strategy can dynamically adjust the decoding process according to the ,network bandwidth conditions to ensure the decoding quality of key areas and ,keyframe sequences.

[0050] Step S300 at least includes steps S310-S330: S310: According to the final decoding strategy, high-resolution decoding is performed on the key areas and key frames in the calibrated image sequence, and high-resolution decoding is performed using an image compression standard to obtain a high-resolution image area.

[0051] According to the final decoding strategy generated in step S220, the key areas in the calibrated image sequence (determined by the context information sequence and the motion state information) need to be decoded at high resolution. The information of the key areas usually comes from the dynamic image features identified by the deep learning model in the S210 stage (such as the motion state of the target area, the image area with significant changes, etc.). The key areas need to maintain high image quality, so an efficient decoding algorithm will be applied during decoding.

[0052] Specifically, assuming that a certain key area is identified in S210 At the moment Position in the image sequence (e.g. ). In this step, an efficient decoding algorithm is used , such as image compression standards (such as JPEG2000, HEVC, etc.), decode the area. For each key area, high-resolution decoding is performed to retain detail information to the greatest extent.

[0053] ; in, represents the decoded high-resolution image area, Represents the image Middle key area The result after applying an efficient decoding algorithm.

[0054] This step extracts the key areas for separate high-resolution decoding based on the recognition results of the image dynamic change pattern in S210. The recognition information of the important areas has been obtained in the S210 stage, so here only the information needs to be used to call an efficient decoding algorithm to process the key areas to ensure that the details of the image are preserved.

[0055] S320: Perform low-resolution decoding on non-key areas and non-key frames in the calibrated image sequence using a compressed sensing technology to obtain a low-resolution image area.

[0056] After completing high-resolution decoding of key areas and key frames in S310, non-key areas and non-key frames need to be processed next. The identification of non-key areas and non-key frames also originates from the decoding strategy generated in S210, and the non-key areas and non-key frames do not need to retain all details at high resolution. The processing of non-key areas and non-key frames will use Compressive Sensing (CS) technology to achieve low-resolution decoding to save bandwidth and computing resources.

[0057] Assume that the non-critical area in the calibrated image sequence is identified in S210 as , the non-critical area can be decoded by compressed sensing technology. Compressed sensing technology compresses the signal of the non-critical area before decoding, and uses low-resolution processing to reduce data transmission requirements. Specifically, the compressed sensing algorithm The low-resolution decoding of non-critical areas can be performed as follows: ; in, is the decoded low-resolution image, which represents the non-key area image after low-resolution decoding. Represents the image in the calibrated image sequence Key Areas in Central Africa Decoding result after applying the compressed sensing algorithm.

[0058] It should be noted that the non-critical area of ​​this step is determined based on the context information sequence and decoding strategy extracted in S210. Compared with the key area in step S310, the non-critical area uses compressed sensing technology for low-resolution decoding to improve decoding efficiency and save network bandwidth.

[0059] S330: Dynamically adjust the image decoding process of the calibrated image sequence according to the network bandwidth status.

[0060] First, the input data of this step mainly comes from the context information sequence obtained in S130 and S230, the final decoding strategy, and the real-time network bandwidth status. In particular, the real-time network bandwidth (derived from the network monitoring system) will serve as a key input to guide adjustments in the decoding process.

[0061] Dynamically adjust the image decoding process according to the network bandwidth conditions. This can be done in the following ways: ; in, It means that the image decoding process is dynamically adjusted according to the network bandwidth conditions. To dynamically adjust the function, and They are the high-resolution and low-resolution image regions respectively.

[0062] For example, when the network bandwidth is low (i.e. If the bandwidth is less than a certain threshold, the transmission volume of the high-resolution area will be reduced or the decoding strategy requirements will be lowered to adapt to the current bandwidth environment. In this process, the decoding process will be adjusted according to the real-time changes in bandwidth, so that the key areas still maintain high quality, while non-key areas are further compressed or the resolution is reduced.

[0063] This step is based on the decoding results in S310 and S320, combined with the real-time status of the network bandwidth. , adjust the image decoding process. The bandwidth information comes from the real-time monitoring of the system. This information will directly affect the image decoding strategy in S310 and S320, thereby ensuring that the image quality and decoding efficiency are optimally balanced under the condition of network bandwidth constraints.

[0064] Step S400 at least includes steps S410-S430: S410: Integrate the decoded high-resolution image area and the low-resolution image area to obtain an integrated image sequence; input the integrated image sequence to a seeker display system to obtain real-time feedback data.

[0065] According to the final decoding strategy generated in S220, the high-resolution image area and low-resolution image areas Merge to get the integrated image sequence , and transmitted to the seeker display system according to system requirements. The specific merging process can be expressed by the following formula: ; in, represents the integrated image sequence; Represents a union.

[0066] It should be noted that the seeker display system refers to a dedicated system for real-time display and processing of target area images. The system receives image sequences generated by optical sensors and image decoding modules, and optimizes image decoding strategies in combination with real-time feedback mechanisms.

[0067] This step is based on the results of the final decoding strategy output of the image sequence in S300 and S310, and combines them into the final image stream, that is, the integrated image sequence. , and display output according to the decoding strategy. The output of the integrated image sequence must be consistent with the decoding strategy, ensuring that the key areas are presented in high resolution and the non-key areas are displayed in low resolution to achieve the purpose of bandwidth optimization.

[0068] S420: According to the real-time feedback data from the system, dynamically adjust the final decoding strategy to further optimize the image output effect.

[0069] First, after the integrated image sequence is output in step S410, the seeker display system will generate real-time feedback data based on the real-time image quality and display effect. The feedback data is an important basis for adjusting the decoding strategy.

[0070] Based on real-time feedback data , the system will dynamically adjust the final decoding strategy to obtain a new decoding strategy. The feedback data contains network bandwidth information and image quality rating , we can optimize the image quality by adjusting the resolution of key areas and non-key areas. The specific dynamic adjustment process can be expressed by the following formula: ; in, Indicates the new decoding strategy adjusted according to real-time feedback data. To dynamically adjust the function, and Represents the real-time network bandwidth and image quality score respectively. By adjusting the final decoding strategy, the image output effect is optimized to ensure that the image reaches the best balance under the current network bandwidth and image quality score requirements.

[0071] This step is based on the integrated image sequence output in S410 , combined with real-time feedback data , adjust the decoding strategy. The result of dynamic adjustment will affect the next decoding strategy and the quality output of subsequent images.

[0072] S430: Continuously monitor the image decoding process in the image sequence, obtain monitoring data including image delay and bandwidth usage; and adjust the new decoding strategy based on the monitoring data.

[0073] First, after obtaining a new decoding strategy in step S420, the system will continue to monitor the decoding process of the image and evaluate the image quality and bandwidth consumption. Real-time monitoring involves multiple aspects such as image decoding delay, image loss, and decoding accuracy.

[0074] In order to ensure continuous optimization of the image decoding process, the system needs to regularly (or in real time) adjust new decoding strategies based on monitoring data to ensure the output quality and real-time performance of the image. and bandwidth usage , then adjust the function The adjustment can be performed as follows. The adjustment process can be expressed as: ; in, Indicates the decoding strategy adjusted according to the monitoring data. For decoding delay, is the bandwidth usage, It is a new decoding strategy in S420.

[0075] This step is based on the new decoding strategy in S420 to further optimize the image decoding process. Through continuous monitoring and adjustment, it is ensured that the decoding strategy always maintains the best performance under changing network conditions and image quality requirements.

[0076] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing the embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for optimizing an image decoding process using context information, characterized in that: The method comprises: Acquire an image sequence and environmental status data in real time, wherein the image sequence includes more than one continuous frame image; preprocess the image sequence and calibrate the moving target to obtain the motion state of the moving target area, and generate a calibrated image sequence; construct a context information sequence according to the calibrated image sequence and environmental status data; Based on the context information sequence, a deep learning model is used to analyze the dynamic change pattern of the calibrated image sequence, identify key areas, non-key areas, key frame sequences and non-key frame sequences, and generate a final decoding strategy in combination with the motion state of the target area; According to the final decoding strategy, the key area and key frame sequence are decoded at high resolution to obtain a decoded high-resolution image area; the non-key area and non-key frame sequence are decoded at low resolution to obtain a decoded low-resolution image area; and the image decoding process of the calibrated image sequence is dynamically adjusted according to the network bandwidth status; The decoded high-resolution image area and low-resolution image area are integrated to obtain an integrated image sequence; the integrated image sequence is input into a seeker display system to obtain real-time feedback data; and the final decoding strategy is adjusted based on the real-time feedback data to further optimize the image decoding process.

2. The method for optimizing the image decoding process by using context information according to claim 1, characterized in that: The step of acquiring the image sequence and environmental status data in real time also includes: Continuous frame images are acquired from an image acquisition device in real time and form an image sequence; environmental status data associated with the image sequence is acquired from an environmental monitoring device, the environmental status data including: lighting conditions, weather conditions, relative movement speed and / or temperature of the target.

3. The method for optimizing the image decoding process by using context information according to claim 2, characterized in that: The step of generating a calibrated image sequence further includes: Performing denoising processing on the image sequence using an image denoising algorithm to obtain a preprocessed image sequence; The moving target in each frame of the preprocessed image sequence is located and calibrated by the target detection algorithm, the motion state of the moving target area is obtained, and the calibrated image sequence is generated.

4. The method for optimizing the image decoding process by using context information according to claim 3, characterized in that: The step of constructing a context information sequence specifically includes: Context information is extracted according to the calibrated image sequence and environmental status data, and a context information sequence is constructed in combination with the time sequence; the context information sequence includes the time series features, spatial features and environmental status data of the calibrated image sequence.

5. The method for optimizing the image decoding process by using context information according to claim 4, characterized in that: The step of analyzing the dynamic change pattern in the calibrated image sequence using a deep learning model based on the context information sequence specifically includes: The deep learning model is an LSTM network and / or a CNN network. The LSTM network is used to extract time series features and learn the changing trend of the calibrated image sequence in the time dimension. The CNN network is used to extract spatial features and detect moving targets and key areas in the calibrated image sequence. The area with significant dynamic changes in each frame of the identified calibrated image sequence is regarded as the key area, and the area other than the key area is regarded as the non-key area; The frame sequences with significant dynamic changes are identified as key frame sequences, and the frame sequences other than the key frame sequences are identified as non-key frame sequences.

6. The method for optimizing the image decoding process by using context information according to claim 5, characterized in that: The step of generating the final decoding strategy comprises: Based on the key area, the non-key area, the key frame sequence and the non-key frame sequence, and in combination with the motion state of the target area, a preliminary decoding strategy is generated; According to the real-time network bandwidth status and image transmission requirements, the preliminary decoding strategy is adjusted to generate a final decoding strategy.

7. The method for optimizing an image decoding process by using context information according to claim 6, characterized in that: The step of performing high-resolution decoding on the key area and key frame sequence according to the final decoding strategy to obtain a decoded high-resolution image area also includes: The key area and key frame are decoded with high resolution by using the standard image compression algorithm to obtain the high resolution image area; Compressed sensing technology is used to perform low-resolution decoding on non-key areas and non-key frames to obtain low-resolution image areas.

8. The method for optimizing an image decoding process by using context information according to claim 7, characterized in that: The adjusting the final decoding strategy based on the real-time feedback data further includes: The final decoding strategy is adjusted based on the real-time feedback data of the seeker display system, the real-time feedback data including the network bandwidth status and / or the image quality score, to obtain a new decoding strategy.

9. The method for optimizing the image decoding process by using context information according to claim 8, characterized in that: The step of adjusting the final decoding strategy based on the real-time feedback data further includes: Continuously monitor the image decoding process to obtain monitoring data including image delay and bandwidth usage; adjust new decoding strategies based on the monitoring data.

Images