Inter-frame image code rate control method based on KAN network and electronic equipment

Abstract

The invention relates to an inter-frame image code rate control method based on a KAN network and electronic equipment, and the method comprises the following steps: obtaining a medical image video, and extracting the inter-frame dynamic features of the medical image video; constructing a KAN network by using the extracted dynamic features, and parameterizing by using a B spline; training a KAN network and predicting an image code rate of a next frame; and dynamically adjusting the image code rate of each frame based on the predicted image code rate, and adjusting the image code rate in combination with the coding quality and the real-time feedback of the buffer area state. Compared with the prior art, the method has the advantages of realizing efficient and accurate code rate control, using coding requirements of different complex scenes, reducing calculation complexity and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of video image processing, and in particular to an inter-frame image bit rate control method and electronic device based on a KAN network. Background Art

[0002] Video compression technology plays an increasingly important role in medical applications. With the explosive growth of medical imaging data, such as high-resolution MRI and CT scans, video compression has become a key technology to reduce the cost of data storage and transmission. These technologies not only enable medical images and videos to be transmitted more efficiently while maintaining high quality, but also promote the development of telemedicine and electronic health records. The demand for real-time video transmission has prompted the continuous optimization of compression algorithms to ensure reliable image quality under low bandwidth conditions. The application of AI has accelerated the advancement of video compression technology, making intelligent compression and content recognition possible, further improving the efficiency and accuracy of medical services. In the future, video compression technology will continue to develop in a more efficient, more precise and safer direction, bringing more innovation and convenience to the medical industry.

[0003] The current video compression rate control methods include buffer control, which adjusts the bit rate by managing the buffer to balance bandwidth and video quality; perception-based control methods, which dynamically adjust the bit rate based on the human eye's perception of video quality; and content-based control methods, which optimize the bit rate allocation based on the characteristics of the video content. The buffer control method can effectively avoid frame loss and improve video quality, but it increases the implementation complexity and latency. The perception-based control method can provide a better user experience, but requires complex algorithms and calculations. The content-based control method can perform fine-grained control based on the characteristics of the video content, but requires accurate content analysis and recognition technology. Choosing an appropriate rate control method requires considering the specific needs of the application scenario, such as the balance of latency requirements, network stability, and user experience.

[0004] How to achieve efficient encoding of video images and reduce computational complexity has become a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a method and electronic device for controlling the inter-frame image bit rate based on a KAN network in order to overcome the defects of the prior art.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to one aspect of the present invention, a method for controlling an inter-frame image bit rate based on a KAN network is provided, the method comprising the following steps:

[0008] Step S1, obtaining a medical image video and extracting its inter-frame dynamic features;

[0009] Step S2, constructing a KAN network using the extracted dynamic features and parameterizing it using B-splines;

[0010] Step S3, training the KAN network and predicting the image bitrate of the next frame;

[0011] Step S4, dynamically adjusting the image bit rate of each frame based on the predicted image bit rate, and adjusting the image bit rate in combination with real-time feedback of encoding quality and buffer status.

[0012] Preferably, the predicted image bit rate of the next frame is specifically:

[0013] The KAN network predicts the image bitrate of the next frame based on the input dynamic feature information entropy H(X).

[0014]

[0015] H(X)=-∑ i p i logp i

[0016] Among them, f(.) is the prediction function in the KAN network, p i is the probability distribution of the dynamic feature X, H(X) is the information entropy of the dynamic feature X, and t is the current time.

[0017] Preferably, the KAN network uses mean square error as the loss function, specifically:

[0018]

[0019] in, is the loss function, which is used to quantify the error between the predicted value and the true value; N is the total number of samples; f(H(X i )) is the KAN network input dynamic feature X i The information entropy H(X i ) is the predicted image bit rate; f(.) is the prediction function in the KAN network; Y i is the actual inter-frame image bit rate of the i-th sample.

[0020] More preferably, the KAN network minimizes the loss function through a gradient descent optimization algorithm and updates the B-spline coefficients.

[0021] Preferably, the dynamic adjustment of the image bit rate of each frame is specifically as follows:

[0022]

[0023] Among them, R t and R t-1 are the actual inter-frame image bit rates at time t and t-1 respectively; α is the smoothing coefficient, which is used to control the weight between the current prediction value and the historical bit rate; Represents the inter-frame image bit rate prediction value at time t.

[0024] Preferably, in step S4, adjusting the image bit rate in combination with real-time feedback of encoding quality and buffer status includes:

[0025] Assume that the adjusted image bit rate is R′ t , get the current image bit rate R t And feedback information, where the feedback information includes feedback information on encoding quality and buffer status, calculate the bit rate adjustment factor ΔR:

[0026]

[0027] R′ t =R t (1+ΔR)

[0028] Among them, α and β are the weights of encoding quality and buffer status respectively, Q t and Q target are the current value and target value of the coding quality, B t and B target They are the current and target values ​​of the buffer state, respectively.

[0029] Preferably, the dynamic features include motion vectors, texture features, inter-frame differences and time domain features.

[0030] More preferably, an optical flow method is used to calculate motion vectors between adjacent video frames; and a texture analysis method is used to extract texture features between frames.

[0031] More preferably, the inter-frame difference refers to the difference in pixel values ​​between adjacent video frames; and the temporal feature includes inter-frame brightness changes.

[0032] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1) This method extracts dynamic features between video frames, uses the KAN network to predict the image bitrate of the next frame, and dynamically adjusts the image bitrate of video encoding according to the historical bitrate. It adjusts the image bitrate in real time based on the real-time feedback of encoding quality and buffer status to achieve efficient and accurate bitrate control and adapt to the encoding requirements of different scenarios.

[0035] 2) The present invention calculates the dynamic feature information entropy and inputs it into the KAN network to predict the image bitrate of the next frame, which can effectively capture and represent the dynamic behavior and relationship of complex systems. Therefore, this method can maintain high resolution in key areas by learning the dynamic behavior of complex systems, and is suitable for image bitrate control in different complex scenes such as medical images.

[0036] 3) The present invention dynamically adjusts the image bit rate of video encoding according to the historical bit rate, and timely adjusts the image bit rate in combination with a real-time feedback mechanism, thereby improving the efficiency and quality of video encoding.

[0037] 4) The KAN network of the present invention simplifies the complexity of the problem, reduces the computational complexity, improves the efficiency of bit rate adjustment, reduces bandwidth requirements and storage space occupancy, and enhances the robustness and adaptability of the system by decomposing a complex multivariable function into multiple one-dimensional functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the inter-frame image bit rate control method based on the KAN network of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] This embodiment relates to a method for inter-frame image bit rate control based on a KAN network, which extracts dynamic features between video frames (such as motion vectors, textures, inter-frame differences, and time domain features), uses the Kolmogorov-Arnold Networks (KAN) network for modeling and prediction, and dynamically adjusts the bit rate of video encoding. This method optimizes bit rate allocation in real time by calculating feature information entropy and combining feedback control, thereby improving the efficiency and quality of video encoding. Ultimately, the KAN network achieves efficient and accurate bit rate control by learning the dynamic behavior of complex systems to adapt to the encoding requirements of different scenarios.

[0041] KAN is based on the Kolmogorov-Arnold representation theorem, which states that any continuous multivariable function can be represented as a superposition of a finite number of single-variable functions. The KAN network simplifies the complexity of the problem by decomposing complex multivariable functions into multiple one-dimensional functions. In the KAN network, each one-dimensional function is usually parameterized by B-Splines, which consist of a set of control points and basis functions, and the parameters are optimized through a training process. During the training process, the back-propagation algorithm and loss function are used to adjust the network parameters to minimize the prediction error. Through this structure, the KAN network can effectively capture and represent the dynamic behaviors and relationships of complex systems. Its advantage is that it improves the expressive power and computational efficiency of the model, especially for processing high-dimensional data. Ultimately, the KAN network achieves accurate modeling and prediction of the behavior of complex systems, making it perform well in various application scenarios.

[0042] like Figure 1 , the method consists of four steps:

[0043] Step S1, video frame-to-frame dynamic feature extraction: First, extract frame-to-frame dynamic features from the acquired medical image video sequence, including motion vectors, texture features, frame-to-frame differences, and time domain features. These features are used to describe the changes and complexity between video frames and provide input for subsequent bit rate prediction.

[0044] Step S2, constructing a KAN network: constructing a KAN network using the extracted dynamic features and parameterizing it using B-splines.

[0045] Step S3, training the KAN network and predicting: Through the training process, the parameters of the KAN model are adjusted so that it can accurately predict the image bitrate of the next frame. The loss function (such as mean square error) and back propagation algorithm are used in the training process to minimize the prediction error.

[0046] Step S4, dynamic bit rate allocation of inter-frame images: dynamically adjust the compression bit rate of each frame according to the inter-frame image bit rate predicted by the KAN model.

[0047] Step S5, real-time optimization strategy: Combined with the real-time feedback control mechanism, the bit rate allocation strategy is further optimized to ensure the quality and efficiency of video encoding in different scenarios. This step ensures the adaptability and robustness of video encoding and improves the overall encoding performance.

[0048] In step S1, inter-frame dynamic features in the video sequence are extracted, including motion vectors, texture features, inter-frame differences and time domain features.

[0049] Motion Vector: Use optical flow to calculate the motion vector between adjacent video frames.

[0050] v(x,y) = (u(x,y),v(x,y)) (1)

[0051] Where v(x,y) represents the motion vector at position (x,y), and u(x,y) and v(x,y) are the motion components in the horizontal and vertical directions, respectively.

[0052] Texture features: Use texture analysis methods (such as gray-level co-occurrence matrix) to extract texture features between frames.

[0053] T = GLCM(I) (2)

[0054] Among them, T is the texture feature matrix, I is the image gray value, and GLCM is a statistical model function constructed by counting the co-occurrence pattern of pixel gray values. It can be widely used in image processing tasks such as texture classification, segmentation, retrieval and recognition.

[0055] Frame Difference: Calculates the difference in pixel values ​​between adjacent video frames.

[0056] ΔI t =I t+1 -I t (3)

[0057] Among them, I t and I t+1 are the pixel values ​​of the image frames at time t and t+1 respectively.

[0058] Temporal features: Calculate the temporal features between frames, such as the brightness changes between frames.

[0059]

[0060] Among them, F t represents the average brightness of the frame at time t, I t (x i ,y i ) is the position in the frame (x i ,y i ) is the pixel value, and N is the total number of pixels.

[0061] In step S2, a KAN network model is established and parameterized using B-Splines. According to the KolmogorovArnold representation theorem, any continuous function f:[0,1] n →R can be expressed as a combination of a series of single variable continuous functions:

[0062]

[0063] where x1,x2,…x nThe independent variable defined for the input variable is in the interval [0,1]; p represents the index in the decomposition, which is used to count the combination of univariate functions in the Kolmogorov-Arnold representation; Φ q It is usually called the outer function, whose input is p Apply a set of inner functions The linear (summation) combination result after q is used to enumerate a set of univariate functions (2n+1 in total) required in the Kolmogorov-Arnold representation to ensure the representation of any continuous function.

[0064] In the KAN network, the univariate function Expressed using B-spline:

[0065]

[0066] Among them, c q,p,i are the learnable B-spline coefficients, B i,k is the B-spline basis function; i is the index of the basis function, i.e., used to traverse all B-spline basis functions in the summation. k is generally used to represent the order or number-related parameters of the B-spline basis function, so that the summation range is i=0 to i=k.

[0067] In step S3, the extracted dynamic features are used to train the KAN network to predict the image bitrate of the next frame. Assuming that the information entropy of the dynamic feature X is H(X), the calculation formula for the information entropy is:

[0068] H(X) = -∑ i p i logp i (7)

[0069] Among them, p i is the probability distribution of dynamic features.

[0070] The KAN network predicts the image bitrate of the next frame based on the input dynamic feature information entropy H(X). Where f(.) is the prediction function in the KAN network:

[0071]

[0072] When training the KAN model, the mean square error (MSE) is used as the loss function:

[0073]

[0074] in, Loss function, used to quantify the error between the model prediction value and the true value; N: total number of samples, indicating the number of samples in the training data; The sum operation accumulates from the 1st sample to the Nth sample. f(H(X i )): The predicted value of the model. Specifically, H(X i ) represents the input feature X i The information entropy of Y is , and f is the prediction function of the KAN model, which makes predictions based on the information entropy of the input features. i : The true value of the i-th sample, in this example, Y i is the actual inter-frame image bit rate. (f(H(X i ))-Y i ) 2 : Squared error, which represents the square of the difference between the predicted value and the true value of the i-th sample.

[0075] The loss function is minimized through optimization algorithms such as gradient descent, and the B-spline coefficients are updated.

[0076] In step S4, the bit rate of the inter-frame images is dynamically allocated.

[0077] According to the predicted inter-frame image bitrate, dynamically adjust the image bitrate of each frame:

[0078]

[0079] Among them, R t and R t-1 are the actual inter-frame image bit rates at time t and t-1 respectively; α is the smoothing coefficient, which is used to control the weight between the current prediction value and the historical bit rate; Represents the inter-frame image bit rate prediction value at time t.

[0080] In step S5, a real-time optimization strategy is implemented combining encoding quality and buffer status.

[0081] Combined with real-time monitoring and feedback mechanism, the image bit rate allocation strategy is adjusted in real time according to the current inter-frame dynamic characteristics to ensure the quality and efficiency of video encoding in different scenarios.

[0082] Assume that the adjusted image bit rate is R′ t , get the current bit rate R t and feedback information (quality Q t and buffer status B t ). Then calculate the bit rate adjustment factor:

[0083]

[0084] Update bitrate:

[0085] R′ t=R t ·(1+ΔR) (12)

[0086] Among them, α and β are the weights of quality and buffer state respectively, Q t and Q target are the current value and target value of quality, B t and B target They are the current and target values ​​of the buffer state, respectively.

[0087] The optimization function dynamically adjusts the bitrate of video encoding by combining the feedback of encoding quality and buffer status, thereby maintaining a stable buffer status while ensuring video quality. This method can significantly improve the adaptability and robustness of encoding in complex video scenarios.

[0088] This inter-frame image bitrate control method based on the KAN network extracts dynamic features between video frames (including motion vectors, textures, inter-frame differences, and temporal features) and calculates the confidence information entropy of these features to provide accurate and effective input data for bitrate prediction; it uses the B-spline parameterized KAN model for modeling and prediction, and combines the real-time feedback control mechanism to dynamically adjust the video encoding bitrate according to the prediction results of the KAN model, ensuring efficient bitrate control and optimized video encoding quality under various network conditions and application scenarios. This method is particularly effective when processing complex video scenes and can significantly improve the adaptability and robustness of encoding.

[0089] Video rate control has significant benefits in medical applications, especially in the fields of medical imaging and telemedicine. First, by optimizing the bit rate, the amount of data transmitted can be reduced while maintaining image quality, thereby reducing bandwidth requirements. This is particularly important in telemedicine diagnosis, as efficient data transmission can ensure real-time and high-quality diagnosis. Second, rate control can help compress and store large amounts of medical imaging data, saving storage space and costs. In addition, stable rate control can ensure the smoothness of video streaming and avoid diagnostic delays caused by network fluctuations. Dynamic bit rate adjustment of key image areas helps to retain high resolution in important details and improve diagnostic accuracy. Furthermore, reducing the amount of transmitted data can also enhance data security and privacy protection, reducing the risk of sensitive information exposure. Overall, rate control technology has brought significant improvements to the provision and management of medical services by improving efficiency and quality.

[0090] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0091] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0092] The processing unit performs the various methods and processes described above. For example, in some embodiments, the method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the CPU can be configured to perform the method in any other appropriate manner (e.g., by means of firmware).

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for controlling inter-frame image bit rate based on a KAN network, characterized in that: The method comprises the following steps: Step S1, obtaining a medical image video and extracting its inter-frame dynamic features; Step S2, constructing a KAN network using the extracted dynamic features and parameterizing it using B-splines; Step S3, training the KAN network and predicting the image bitrate of the next frame; Step S4, dynamically adjusting the image bit rate of each frame based on the predicted image bit rate, and adjusting the image bit rate in combination with real-time feedback of encoding quality and buffer status.

2. The inter-frame image bit rate control method based on the KAN network according to claim 1, characterized in that: The predicted image bit rate of the next frame is specifically: The KAN network predicts the image bitrate of the next frame based on the input dynamic feature information entropy H(X). H(X)=-∑ i p i logp i Among them, f(.) is the prediction function in the KAN network, p i is the probability distribution of the dynamic feature X, H(X) is the information entropy of the dynamic feature X, and t is the current time.

3. The inter-frame image bit rate control method based on the KAN network according to claim 1, characterized in that: The KAN network uses mean square error as the loss function, specifically: in, is the loss function, which is used to quantify the error between the predicted value and the true value; N is the total number of samples; f(H(X i )) is the KAN network input dynamic feature X i The information entropy H(X i ) is the predicted image bit rate; f(.) is the prediction function in the KAN network; Y i is the actual inter-frame image bit rate of the i-th sample.

4. The inter-frame image bit rate control method based on the KAN network according to claim 3 is characterized in that: The KAN network minimizes the loss function through a gradient descent optimization algorithm and updates the B-spline coefficients.

5. The inter-frame image bit rate control method based on the KAN network according to claim 1, characterized in that: The dynamic adjustment of the image bit rate of each frame is specifically as follows: Among them, R t and R t-1 are the actual inter-frame image bit rates at time t and t-1 respectively; α is the smoothing coefficient, which is used to control the weight between the current prediction value and the historical bit rate; Represents the inter-frame image bit rate prediction value at time t.

6. The inter-frame image bit rate control method based on the KAN network according to claim 1, characterized in that: In the step S4, adjusting the image bit rate in combination with real-time feedback of the encoding quality and buffer status includes: Assume that the adjusted image bit rate is R t ', get the current image bit rate R t And feedback information, where the feedback information includes feedback information on encoding quality and buffer status, calculate the bit rate adjustment factor ΔR: R t '=R t ·(1+ΔR) Among them, α and β are the weights of encoding quality and buffer status respectively, Q t and Q target are the current value and target value of the coding quality, B t and B target They are the current and target values ​​of the buffer state, respectively.

7. The inter-frame image bit rate control method based on the KAN network according to claim 1, characterized in that: The dynamic features include motion vectors, texture features, inter-frame differences and time domain features.

8. The inter-frame image bit rate control method based on the KAN network according to claim 7, characterized in that: The optical flow method is used to calculate the motion vector between adjacent video frames; the texture analysis method is used to extract texture features between frames.

9. The inter-frame image bit rate control method based on the KAN network according to claim 7, characterized in that: The inter-frame difference refers to the difference in pixel values ​​between adjacent video frames; the temporal features include inter-frame brightness changes.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 9 is implemented.

Images