The application discloses an intelligent screen
interaction method and
system based on a
large model, and belongs to the technical field of intelligent
home screen interaction. The method comprises the following steps: collecting data to obtain multi-
modal data; analyzing the multi-
modal data according to an instruction corpus to obtain an interaction instruction, and identifying a
spatial interaction position corresponding to the interaction instruction according to the multi-
modal data; determining a
target control instruction according to the
spatial interaction position and the interaction instruction; and performing control adjustment according to the
target control instruction. The method breaks through the limitations of existing active instruction triggering, realizes the accurate identification of non-standard instructions such as self-talk, fragmented speech and complaints in a home scene through non-standard instruction recognition and implicit
demand analysis, fits the natural and casual behavior habits of home life, and greatly improves the intelligent level of interaction. Through
spatial cognition, automatic division of a
virtual property space in the home and spatial context analysis are realized, the
semantic difference problem of the same instruction in different spaces is solved, and the spatial
adaptive response of the instruction is realized.