The method comprises: in the first, respectively according to the frame layer rate controlling policy, head information predication model, and complexity summation linear predication model, calculating the target bits number of all macro blocks in current frame, head information bit number, and predicating the complexity summation; in the second , calculating the target bit number of encoded macro blocks in current frame, head information bit number and complexity summation; according to the code transformation rate-distortion model, calculating the quantization step length of current macro blocks, and making the rate-distortion optimal encode for the current macro blocks; according to the actual encode data of current macro block, updating the target bit number, head information bit number and complexity summation of the encoded macro block; deciding if the current macro block is the last one of current frame; if not, then updating the coefficient of the encoded signal source rate-distortion model; repeating above step, until last macro block; if yes, then updating the complexity summation and coefficient of the linear predication model.