Conjugated diene polymers having good wear resistance, mechanical properties, storage stability, processability and a reduced cold flow are produced by polymerizing a conjugated diene compound with a catalyst of rare earth element compound, a compound containing at least one halogen atom, an aluminoxane, and an organoaluminum compound in an organic solvent and reacting the resulting polymer just after the polymerization with at least one compound selected from the group consisting of a halogenated organometallic compound, a halogenated metal compound, an organometallic compound, a heterocumulene compound, a hetero three-membered-ring containing compound, a halogenated isocyano compound, a carboxylic acid, an acid halide, an ester compound, a carbonic ester compound, an acid anhydride and a metal salt of a carboxylic acid as a modifying agent.