The embodiment of the invention discloses an annotation object control device. According to the device, after annotation boxes are adopted to annotate objects in an image, control points of the annotation boxes are acquired, and a corresponding relation between the control points and the objects is established; if a mouse clicking event is monitored, a mouse position is acquired; and if it is known that the mouse position is in at least two annotation boxes through judgment, target control points closest to the mouse position in the annotation boxes are acquired, and target objects corresponding to the target control points are obtained according to the corresponding relation. According to the embodiment, the mouse position is acquired after the mouse clicking event is monitored, the target control points closest to the mouse position are calculated, and the target objects corresponding to the target control points are obtained according to the corresponding relation between the control points and the objects. Therefore, the target objects can be determined according to the mouse clicking position, the number of interactions between an annotation tool and a user is reduced, the annotation process is simpler, and meanwhile annotation time is saved.