Learning type interactive game projection system

Abstract

The invention discloses a learning type interactive game projection system, and relates to the technical field of projection interaction. The system comprises an environment perception module which collects images, depth and voice to form environment perception data, an interaction load construction module which generates an interaction load coefficient based on the environment perception data, and an identity and behavior management module which generates an identity label and a behavior record in combination with the environment perception data and a historical task record. The task configuration module generates a task parameter set and scene configuration accordingly, the projection control module forms an image control signal and an audio control signal, and the interaction detection module generates an interaction event and an interaction track and updates historical task records and user archives. Through environment information association management and an interactive load driven dynamic task configuration and presentation linkage mechanism, the interactive game projection system has the capability of real-time adaptation and accurate recording of an interactive state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of projection interaction, in particular to a learning type interactive game projection system. BACKGROUND

[0002] The interactive projection system is widely used in education and training, entertainment experience and immersive interactive scenes. The existing system usually collects user action information through a camera device, obtains spatial position changes through a depth sensor, and acquires voice trigger information through audio acquisition hardware, and then combines a preset task flow to drive the projection device to complete image display and audio output. However, the existing system has certain limitations in the data processing link and the task updating mechanism.

[0003] The existing interactive projection system usually takes image frames, depth frames and voice signals as independent data sources in the environment information acquisition stage, lacks unified management of time markers and space markers, and is difficult to ensure the consistency of multiple types of data in the subsequent processing process. In the interactive behavior analysis process, the perception of user action changes, voice trigger frequency and spatial trajectory changes usually stays in single-dimensional statistics, and lacks comprehensive quantitative expression methods that can reflect the change of interaction intensity, making it difficult for the task flow to be dynamically adjusted according to the interaction state.

[0004] The existing system mainly relies on fixed numbers or single recognition results in identity management and behavior record maintenance, lacks identity consistency verification mechanism across time periods, and lacks continuity management of user behavior characteristics in different scenes or different sessions, resulting in difficulty in processing task allocation based on long-term behavior characteristics. In the task configuration stage, the existing system is more based on static templates to build task parameters, and is difficult to adjust in real time according to user action changes, voice trigger state and spatial behavior, thereby affecting the flexibility of task execution.

[0005] In the projection presentation and audio control stage, the existing system usually fixes the output rhythm and presentation method, and lacks the ability to adjust the presentation rhythm or feedback frequency according to the change of interaction intensity. In the interaction detection stage, the existing system usually uses fixed thresholds to judge trigger events, lacks a method of joint analysis combining image control signals, spatial depth information and voice trigger state, resulting in insufficient event recognition accuracy, insufficient trajectory record refinement, and insufficient timely task record updating.

[0006] Therefore, there is an urgent need for a system structure that can realize environment data association management, identity and behavior unified modeling, interaction intensity quantitative expression, task configuration dynamic adjustment, image and audio control rhythm linkage, and interaction record real-time updating, to improve the task configuration capability and interaction detection capability in the interactive projection scene. SUMMARY

[0007] Based on the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a learning interactive game projection system to solve the above technical problems.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solutions: a learning interactive game projection system, comprising: An environment perception module acquires user image information, spatial depth information and voice information to form environment perception data; An interaction load construction module forms an interaction load coefficient representing interaction density, spatial trajectory change amount and voice trigger frequency per unit time based on the environment perception data; An identity and behavior management module generates identity data and behavior record data based on the environment perception data and historical task records; A task configuration module generates task parameter sets and scene configuration data according to the identity data, behavior record data and interaction load coefficient; A projection control module forms image control signals and audio control signals to drive the projection device and audio device to work according to the task parameter sets, scene configuration data and interaction load coefficient; An interaction detection module generates interaction event data and interaction trajectory data according to the image control signals, environment perception data and interaction load coefficient, and updates the historical task records and user archives.

[0009] The present application further provides that the environment perception module comprises: Image frames containing user contour information and motion contour information are acquired in the projection coverage area to form image data; Spatial depth information is acquired under the time reference corresponding to the image frames to form spatial depth data; Voice signals in the user interaction process are acquired in the projection coverage area to form voice data; The image data, spatial depth data and voice data are associated and sorted according to the unified time mark and spatial mark to form the environment perception data.

[0010] The present application further provides that the interaction load construction module comprises: The interaction behaviors generated by the user based on the environment perception data are time-slice divided in the continuous time interval to form an interaction behavior set corresponding to the time interval; The interaction behavior set is counted in each time interval to generate an interaction density parameter corresponding to the time interval; The spatial position change information in the environment perception data is sorted in the corresponding time interval to form a spatial trajectory change amount parameter; The voice data in the environment perception data is counted in the corresponding time interval to form a voice trigger frequency parameter; An interaction load coefficient is formed based on the interaction density parameter, the spatial trajectory change parameter and the voice trigger frequency parameter.

[0011] The application is further configured to generate the identity identification data, including: A target area is established based on facial image information in the environment perception data, and a unique identity feature segment is formed. In the continuous collection interval, the identity feature segment is subjected to feature consistency inspection to form stable identity expression data. The stable identity expression data is compared with user number information in the historical task record to form the identity identification data.

[0012] The application is further configured to generate the behavior record data, including In the continuous time period, spatial action change information is extracted based on the environment perception data to form an action segment sequence. In the action segment sequence, a behavior change moment is identified to form behavior change data. The behavior change data is subjected to time sequence correlation processing based on the historical task record to form the behavior record data.

[0013] The application is further configured to generate the task parameter set, including: Task matching conditions are established based on the identity identification data to form task matching information. Behavior continuity and behavior change position are analyzed based on the behavior record data to form behavior structure information. Task condition reorganization is completed based on the interaction load coefficient, the task matching information and the behavior structure information to form the task parameter set.

[0014] The application is further configured to generate the scene configuration data, including: A scene display interval is determined based on the task parameter set to form scene area data. Scene preference data is extracted based on the identity identification data and the behavior record data to form a scene preference structure. The scene configuration data is generated based on the integration of the scene area data and the scene preference structure.

[0015] The application is further configured to form the image control signal, including: An image presentation interval and task execution stage information are determined based on the task parameter set to form an image stage structure. A scene element layout relationship is extracted based on the scene configuration data to form an image layout structure. The display rhythm of the image stage structure and the image layout structure is adjusted based on the interaction load coefficient to form the image control signal.

[0016] The application is further configured that the formation of the audio control signal comprises the following steps: extracting task prompt associated data based on the task parameter set, and forming an audio prompt structure; identifying scene acoustic elements based on the scene configuration data, and forming an audio scene structure; adjusting the output frequency and timing of the audio prompt structure and the audio scene structure based on the interaction load coefficient, and forming the audio control signal.

[0017] The application is further configured that the interaction detection module comprises: identifying the trigger position relationship between the user and the projection content based on the image control signal, and forming initial interaction event data; analyzing the action path information of the user in the space based on the environmental perception data, and forming initial interaction trajectory data; adjusting the sampling granularity and recording density of the initial interaction event data and the initial interaction trajectory data based on the interaction load coefficient, and forming interaction event data and interaction trajectory data; updating the historical task record and the user profile based on the interaction event data and the interaction trajectory data.

[0018] The application provides a learning type interactive game projection system, and the beneficial effects of the system include: 1. A unified processing mechanism of environmental information association: through unified marking and associated arrangement of image data, spatial depth data and voice data, the environmental perception information forms a structured data set, which is beneficial to maintaining data consistency in identity management, behavior analysis and interaction detection stages, and reducing recognition errors caused by information isolation; 2. Dynamic task configuration capability based on interaction load coefficient: through the interaction load coefficient composed of interaction density, spatial trajectory change amount and voice trigger frequency, the task parameter set and the scene configuration data can be dynamically adjusted according to the interaction intensity change, the adaptation capability of the task configuration stage to the user behavior change is improved, and the flexibility of the system in different interaction states is enhanced; 3. Linkage adjustment mechanism of projection presentation and interaction detection: by introducing the interaction load coefficient in the formation process of the image control signal and the audio control signal, the image presentation rhythm and the audio feedback output can be adjusted according to the interaction state change, and at the same time, the event sampling and trajectory recording density are adjusted according to the coefficient in the interaction detection stage, so that the task record and the user profile can be updated in time, and the tracking accuracy of the system to the interaction process is improved.

[0019] The above description is only a summary of the technical scheme of the application, in order to more clearly understand the technical means of the application, the specific embodiments of the application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 The flowchart illustrates an exemplary embodiment of the present invention: a learning-based interactive game projection system. Detailed Implementation

[0021] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0022] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0024] Example 1: A learning-oriented interactive game projection system, such as Figure 1 As shown, it includes: The environmental perception module collects user image information, spatial depth information, and voice information to form environmental perception data. The interaction load construction module generates an interaction load coefficient based on environmental perception data, representing the interaction density, spatial trajectory change, and voice trigger frequency per unit time. The identity and behavior management module generates identity identification data and behavior record data based on environmental perception data and historical task records; The task configuration module generates a task parameter set and scene configuration data according to the identity data, the behavior record data and the interaction load coefficient; The projection control module forms image control signals and audio control signals according to the task parameter set, the scene configuration data and the interaction load coefficient, and drives the projection device and the audio device to work; The interaction detection module generates interaction event data and interaction trajectory data according to the image control signals, the environment perception data and the interaction load coefficient, and updates the historical task record and the user profile.

[0025] The application further provides that the environment perception module comprises: The image frames containing user contour information and motion contour information are acquired in the projection coverage area to form image data; specifically, the image acquisition device is installed above or on the side of the projection coverage area, and the image frames covering the entire projection area are continuously acquired. The system first generates a background reference according to a preset background sample in each image frame, and then compares the current image with the background reference pixel by pixel to determine which area belongs to the foreground area through brightness difference and color difference. The continuously distributed pixel blocks in the foreground area are marked as human candidate areas. The system performs edge detection on each human candidate area, connects the pixel points with obvious brightness changes between adjacent pixels into a closed boundary as the user contour line. Then, the position changes of the contour lines in adjacent image frames are compared to mark the contour segments that have displacement or deformation, and these contour segments are summarized as motion contour information to form the image data containing user contour information and motion contour information. The spatial depth information is acquired under the time reference corresponding to the image frames to form spatial depth data; specifically, the depth acquisition device is arranged at a position consistent with the field of view of the image acquisition device, and the corresponding spatial depth information is synchronously acquired when each image frame is acquired. The depth acquisition device gives the distance value of each effective pixel position relative to the device, and the system divides these distance values into several levels according to a preset distance interval, for example, a group of distances close to the device is divided into the first level, the medium distance is divided into the second level, and the farther distance is divided into the third level, and so on. The system establishes a mark consistent with the image frame number for the entire frame of depth data under the same time reference, records the distance level to which each pixel belongs as the corresponding level mark, and finally forms the spatial depth data represented by the combination of time reference, pixel position and distance level; The voice signal in the user interaction process in the projection coverage area is collected to form voice data; specifically, voice collection devices are arranged around the projection area, and sound signals in the environment are continuously collected during the interaction process. The system divides the continuous sound signals into sound segments according to a fixed time length, and marks the starting time and ending time of each sound segment. For each sound segment, the system counts the energy change of the sound in the segment, for example, calculates the maximum value, minimum value and overall change trend of the sound energy in the segment, and judges whether the segment belongs to the user's active voice or voice instruction according to the statistical results. When the system judges that it is a valid voice segment, a voice trigger label and a strength level label are added to the segment to form voice data carrying time information and intensity information; The image data, spatial depth data and voice data are associated and sorted according to unified time labels and space labels to form environment perception data. Specifically, the same time reference is used in the image collection, depth collection and voice collection processes, and time labels are assigned to each image frame, each set of spatial depth data and each voice segment. For image data and spatial depth data under the same time label, according to the installation geometry of the camera and the depth device, the pixel position in the image is converted to the corresponding position in the depth data, so that the same user contour area can find the corresponding distance level label in the depth data. For voice data, the image frame number and depth frame number in the time range are recorded as an association index according to the starting time and ending time of the voice segment. Finally, the image data, spatial depth data and voice data near the same time are bundled and stored as the main index of the time label, and the image data index field, depth data index field and voice data index field are set in them respectively to form environment perception data.

[0026] The application further provides that the interaction load construction module comprises: The interaction behavior generated by the user in a continuous time interval is divided into time segments based on the environment perception data to form an interaction behavior set corresponding to the time interval; specifically, in the projection activity process, the continuous running period is divided into several adjacent time segments based on the time labels carried in the environment perception data, each time segment has a fixed time length or floats within a preset time length range. In each time segment, the interaction action records belonging to the time segment are extracted according to the image change information, depth change information and voice trigger information recorded in the environment perception data, such as touching, waving, stepping, pointing, etc. The interaction action records in the same time segment are merged into an interaction behavior set, and a time segment identifier is established for each interaction behavior set; The interaction behavior set is counted in each time interval to generate an interaction density parameter corresponding to the time interval; specifically, for each interaction behavior set, the number of interaction action items appearing in the set is first counted, and the number is compared with the duration of the corresponding time segment to obtain the compactness of the interaction action appearing in the time segment. On this basis, combinations of different numbers and durations are divided into several density intervals, for example, time segments with fewer interaction action items and scattered distribution are divided into a low density interval, and time segments with more interaction action items and concentrated occurrence are divided into a high density interval. Then, an interaction density level or numerical label is assigned to the time segment according to the belonging density interval as the interaction density parameter corresponding to the time segment; The spatial position change information in the environmental perception data is sorted in the corresponding time interval to form a spatial trajectory change quantity parameter; specifically, in each time segment, the continuous position marks of the user in the time segment are connected in time order to form a trajectory according to the spatial position of the user recorded in the environmental perception data. For the trajectory, first, the number of times of obvious position changes of the user is counted, for example, the number of times of moving from a preset area to an adjacent area, or moving from a front area to a side area; then the distance span of the position changes is classified, and short distance small movements and cross-region movements are distinguished and recorded. By integrating the number of position changes and the position span, the spatial trajectory change is divided into several levels, for example, low change level, medium change level and high change level, and a spatial trajectory change quantity parameter is assigned to the corresponding level to represent the movement activity level of the user in the projection space in the time segment; The number of voice data triggers in the environmental perception data in the corresponding time interval is counted to form a voice trigger frequency parameter; specifically, in each time segment, the voice segment set corresponding to the time segment is extracted from the environmental perception data. First, the background noise segments and the user's active sound segments are distinguished, which can be distinguished by comparing the voice energy change, duration and preset energy threshold; after identifying the effective sound segment, the number of effective sound segments in the time segment is counted, and the voice activity can be divided into low frequency trigger, medium frequency trigger and high frequency trigger levels by combining the duration and energy level of each sound segment. According to the belonging level, a voice trigger frequency parameter is marked for the time segment to reflect the voice participation degree of the user in the time segment; The interaction density parameter, the spatial trajectory change parameter and the voice trigger frequency parameter are converted into uniform grade representation forms according to a pre-set grade rule, for example, the interaction density is divided into several grades, the spatial trajectory change is divided into several grades, and the voice trigger frequency is divided into several grades. Then, a pre-set corresponding relation table is inquired according to the combination relation of the three grades, and different interaction load grades or numerical marks are given to different combinations, for example, when the interaction density grade is high, the spatial trajectory change grade is high and the voice trigger frequency grade is high, a high interaction load coefficient is corresponded, and when the interaction density grade is low and the spatial trajectory change grade and the voice trigger frequency grade are both low, a low interaction load coefficient is corresponded. Through the combination and mapping process, the interaction load coefficient which comprehensively reflects the interaction intensity is formed for each time slice, and is stored in the load data record.

[0027] The application further provides that the generated identity data includes: The face image information in the environment perception data is used to establish a target area and form a unique identity feature segment. Specifically, an image frame containing the upper body of a user is selected from the environment perception data, and a face detection process is performed on each frame. Through the characteristics of brightness distribution, edge shape and facial structure, a rectangular area conforming to the outer contour of the face is located, and the area is taken as the target area. The image content in the target area is normalized, for example, the size and brightness range are unified, and the local texture around the eyes, nose and mouth is extracted in detail. The local texture information after the uniform processing is spliced in a fixed order to form an identity feature segment with a fixed length and a fixed structure, so that the identity feature segments collected at different times for the same user are comparable. The identity feature segments are subjected to feature consistency test in a continuous collection interval to form stable identity expression data. Specifically, a continuous collection interval is selected, and step one is repeated for each image in the time interval to obtain a group of identity feature segments arranged in chronological order. The identity feature segments at adjacent time points are compared in turn, and the proximity of the identity features between two collections is judged by calculating the local texture distribution difference, the key area gray difference and the feature change position number. When the differences between the adjacent multiple identity feature segments are all less than a pre-agreed difference threshold, it is indicated that the identity features of the user remain stable in this period of time. All the identity feature segments in this period of time are summarized, for example, the segment with the smallest difference is selected as a representative segment, or a unified expression is extracted according to the most frequently appearing feature mode to form stable identity expression data. If the difference in some time slices is too large, these time slices are marked as invalid data and do not participate in the generation of the stable identity expression data. Based on the stable identity expression data and the user number information in the historical task record, identity identification data is formed, specifically, in the historical task record, a corresponding reference identity expression data is saved for each registered user, and these data are associated with the corresponding user number one by one. The stable identity expression data obtained in step two is compared with the reference identity expression data in the historical record in turn, the similarity degree of each group of comparison results is calculated by comparing the local texture distribution, the key area gray scale distribution and the overall structure difference item by item. When the difference between a certain historical reference data and the current stable identity expression data is within the preset range, and the proximity degree is maintained in multiple comparisons, the user number corresponding to the historical record is taken as the user number of this time recognition. If the difference value exceeds the allowed range after continuous comparison of all historical reference data, a new user number is allocated for the current user, and the current stable identity expression data is associated with the new number and written into the historical task record. Finally, the matched user number, the current collection time and the current session mark are recorded together to form the identity identification data.

[0028] The application is further provided that the generating behavior record data comprises In a continuous time period, the spatial action change information is extracted based on the environment perception data to form an action segment sequence; specifically, in a continuous time period, the spatial position information and the user contour information in the environment perception data are read at a preset time interval. For adjacent time points, the position, posture and contour deformation degree of the user in the projection area are compared one by one, when the displacement distance between the current position and the last position exceeds the preset spatial threshold, or the contour shape is obviously stretched, bent or lifted, the time point is marked as a spatial action change moment. In the same time period, from the first detection of the spatial action change, the spatial position and contour change of the subsequent several continuous time points are combined into an action segment, and the starting time, ending time and spatial position range of each action segment are recorded. In this way, the action process in the continuous time period is divided into multiple action segments to form an action segment sequence with time sequence and spatial position information; When the behavior change moment is identified in the action segment sequence, the behavior change data is formed; specifically, in the action segment sequence, the change between adjacent action segments is analyzed one by one. First, the difference in duration, the difference in spatial coverage and the difference in action direction between the previous action segment and the next action segment are compared, for example, the previous segment is mainly based on slight movement in place, and the next segment appears large range displacement or from static to fast movement. Then, combined with the rhythm change in the action segment, if the action appears from sporadic trigger to frequent trigger, from low amplitude action to high amplitude action, etc., the junction position between the action segments is marked as the behavior change moment. For each behavior change moment, the occurrence time, the previous action segment feature, the next action segment feature and the change type label are recorded to form a behavior change data entry; Based on the historical task record, the behavior change data is processed in time sequence to form the behavior record data, specifically, after obtaining the behavior change data, each behavior change moment is compared with the task time axis in the historical task record in time sequence. For each behavior change data, according to the occurrence time, the task number in the same time interval is found, the behavior change data is bound with the corresponding task number, and the task category, task progress stage and behavior change type are combined to form a behavior record entry. When multiple behavior change data appear in a task time interval, they are arranged in time sequence to form a behavior record chain reflecting the behavior evolution in the task execution process. For multiple task executions of the same user, the behavior record chains of different tasks are connected in turn according to the task start time to form the behavior record data with task number, occurrence time, behavior change type and associated action segment index.

[0029] The application further provides that the generated task parameter set comprises: Based on the identity data, the task matching condition is established to form the task matching information; specifically, the fields related to the user features are read from the identity data, such as age range, learning stage, historical participation task type, historical task completion condition label and the like. According to the pre-set task matching rules, different age ranges are corresponded to different knowledge ranges, different learning stages are corresponded to different task difficulty level intervals, the task types with more historical participation times are marked as priority task types, and the task types with higher historical completion degree are marked as suitable task types. Combined with these rules, the age range, learning stage, historical task type distribution and historical completion condition of the current user are compared to filter out the task type range suitable for the user's basic level and historical habit, and the corresponding recommended difficulty interval and recommended task duration interval are given to form the task matching information as a whole; Based on the behavior record data, the behavior persistence and the behavior change position are analyzed to form the behavior structure information; specifically, after obtaining the task matching information, the behavior record data is analyzed in time sequence. First, for each historical task, the duration of continuous participation of the user in the task process is counted from the behavior record data, such as the length of the time period of continuous valid action or valid attention, and the number of interruptions in the task process is counted, such as the number of long periods of inaction or frequent deviation from the task area. Then, these indicators are compared in different stages of the task to determine the behavior differences of the user in the beginning, middle and end stages of the task, such as a certain type of user who is active in the early stage of the task and is prone to reduced action or decreased attention in the middle and late stages. According to these analysis results, the behavior characteristics are sorted into the behavior structure information, which includes the behavior persistence level, the position of the task stage where the behavior fluctuation is prone to occur, the adaptability to long or short tasks, etc. The task flow structure can be adjusted according to the behavior characteristics of the user; Based on the interaction load coefficient, the task matching information and the behavior structure information, the task condition reorganization is completed to form the task parameter set; specifically, after the task matching information and the behavior structure information are prepared, the interaction load coefficient is introduced to participate in the task condition reorganization. The interaction load coefficient reflects the interaction density, the spatial trajectory change amount and the voice trigger frequency of the user in a unit of time within a certain period of time. When the coefficient is divided into intervals, the cases of frequent interaction, drastic trajectory change and high voice trigger frequency are classified into the high load interval, and the cases of less interaction or gentle change are classified into the low load interval. During the task condition reorganization process, first, a number of candidate task templates are selected from the task library according to the task matching information, and then the structure of these candidate tasks is adjusted according to the behavior structure information, such as shortening the task duration of the user in the fatigue-prone stage and increasing the task depth of the user in the focused stage. Subsequently, according to the interval to which the current interaction load coefficient belongs, the key parameters of the candidate tasks are adjusted as a whole: in the high load interval, the duration of a single task is reduced, the difficulty level of the task is reduced, and the number of interactive targets presented simultaneously is reduced; in the low load interval, the task execution time is appropriately extended, the difficulty level of the task is increased, or the number of task steps is increased. After the above selection, structure adjustment and load adjustment, the task type, task difficulty, stage division, execution time of each stage, number and type of interactive targets corresponding to each stage, etc. are recorded in the form of fields to form the task parameter set.

[0030] The application further provides that the generation of the scene configuration data comprises: Determine the scene display interval based on the task parameter set, and form scene area data; Specifically, according to the task stage division information recorded in the task parameter set and the execution time of each stage, the display interval on the time axis of the entire projection picture is divided, such as the task preparation stage, the core operation stage and the result feedback stage. Then, according to the target quantity, target type and interaction density requirements in the task parameter set, a plurality of candidate areas are divided on the projection picture in a grid or partition manner, and the area combination relationship participating in the display in different task stages is determined by comparing the target quantity and the number of candidate areas. For the area participating in the display, the area number, area position range, corresponding task stage mark and target carrying capacity mark are recorded to form the scene area data. The area not participating in the display is not allocated the target carrying capacity mark in the scene area data, and only the position mark is reserved; Extract scene preference data based on identity data and behavior record data, and form a scene preference structure; Specifically, the user's age, learning stage and historical participation task type are extracted from the identity data, and the stay time, trigger times and area in the trajectory set under different scene contents are extracted from the behavior record data. For the scene content with higher frequency, longer stay time or more concentrated interaction actions in the historical record, such as cartoon image, geometric figure and natural scene content, the number of occurrences and corresponding behavior indicators are counted, and these statistical results are divided into multiple preference level intervals according to the content category. Then, according to the behavior record of the user in different space areas, the concentration degree of trigger events in the left area, the right area, the upper area and the lower area is counted respectively to determine which areas the user is more inclined to operate. Finally, the content category preference level and the space area preference level are combined to form a scene preference structure, which includes a preferred content category list, a priority mark corresponding to each category, and a space area priority mark associated with the content category; Integrate scene area data and scene preference structure to generate scene configuration data, specifically, after the scene area data and scene preference structure are prepared, the scene area corresponding to each task stage is filled with content and allocated with display strategy according to the task stage order. First, according to the area number and position range recorded in the scene area data, select the content category with higher user preference level from the scene preference structure for matching, for example, assign high-priority content to the area with concentrated behavior, and assign secondary-priority content to the area with less behavior but still with interaction record. For the combination of multiple areas in the same stage, the attention degree in the historical behavior record and the target carrying capacity mark in the task parameter set are compared to determine the target display quantity, target type and change rhythm of each area. Record the content category identifier, display order, refresh rhythm mark and task stage corresponding relationship of each area, and arrange them in the task stage order to form the scene configuration data.

[0031] The application is further configured to form the image control signal, which includes: The image stage structure is formed based on the task parameter set, the image presentation interval and the task execution stage information. Specifically, the task start time, the task end time, the task stage division mark and the duration parameter corresponding to each stage are read from the task parameter set. According to these time parameters, a plurality of continuous image presentation intervals are divided on the whole projection running time axis, such as a preparation guide interval, a core operation interval and a result feedback interval. For each interval, the task difficulty level, the interactive target quantity and the stage priority recorded in the task parameter set are combined to determine the function of the interval, that is, the guide display, the key operation prompt or the result presentation, and different image presentation level marks are allocated to different types of intervals. Through the division process, the continuous intervals on the time axis are corresponded to the task execution stages one by one, and the image stage structure including the stage start and end time, the stage type mark and the presentation level mark is formed. The image layout structure is formed based on the scene configuration data to extract the scene element layout relationship. Specifically, after the image stage structure is determined, the region number, the region position range, the content category identifier and the display priority information of each scene element are read from the scene configuration data. For the elements participating in the display in the same task stage, the elements with high priority are arranged in the central region of the user historical behavior set, and the elements with low priority are arranged in the edge region or the region with less behavior. In the same task stage, for the combination of a plurality of regions, the target carrying capacity mark and the content category type recorded in the scene configuration data are compared to determine the number of elements and the content category that need to be displayed in each region in the stage. Through the screening and sorting, the arrangement order of the scene elements in the horizontal direction and the vertical direction, the occupied range of each region and the content superposition order are recorded as a group of layout description information, and the image layout structure is formed to describe the spatial arrangement relationship of the image elements on the projection plane in each task stage. The display rhythm of the image stage structure and the image layout structure is adjusted based on the interactive load coefficient to form an image control signal, specifically, after the image stage structure and the image layout structure are determined, the interactive load coefficient is introduced to adjust the image display rhythm. Firstly, according to the numerical range of the interactive load coefficient, the current interactive state is divided into three levels of low load, medium load and high load. For each stage in the image stage structure, the interactive load level in the corresponding time period is read, when the load level is high, the image refresh period in the stage is set to a short interval, the single picture stay time is reduced, the number of elements displayed in the single area at the same time is reduced, and the appearance frequency of the key prompt element is increased; when the load level is low, the picture stay time is appropriately prolonged, the background element display proportion is increased, and the picture switching frequency is reduced. In the image layout structure, for the behavior concentration area, the core interactive element is kept appearing in the area continuously under the high load level, and the change frequency is reduced under the low load level, and the picture is kept stable. The adjustment results of the stage time, refresh period, element priority, display order and change frequency are encoded into a series of image control instructions arranged in time sequence, each instruction contains target area number, content number, display start and end time mark and display rhythm mark, and the instruction set constitutes the image control signal, which is executed by the projection device in the running process.

[0032] The application is further provided that the formation of the audio control signal includes the following steps: Task prompt associated data is extracted based on the task parameter set to form an audio prompt structure, specifically, the task stage division, task target type, key node mark and duration information of each stage in the task parameter set are read. Firstly, the task stage is divided into several types such as preparation stage, operation stage and result stage, and the corresponding prompt categories are configured for each type of stage, for example, the preparation stage is associated with the guide type prompt, the operation stage is associated with the operation guidance type prompt, and the result stage is associated with the feedback type prompt. Then, according to the task target type, it is judged whether voice prompt, prompt sound or composite prompt sound is needed, for example, it is judged whether the task contains time limit, step limit or accuracy requirement, and additional prompt categories are added to the stage with such limits. For each task stage, according to the stage duration, a plurality of time slices are divided, and the trigger time and duration mark of the target prompt category are allocated in each time slice. Through the above process, the audio prompt structure is formed in the order of the task stage, and the stage number, prompt category, trigger time slice and prompt duration are recorded in the audio prompt structure; The scene acoustic elements are identified based on scene configuration data to form an audio scene structure; specifically, scene elements associated with sound are filtered from the scene configuration data, such as elements marked as water flow, collision, button trigger, reward presentation, etc. For each type of scene element, a set of audio materials matching the element is pre-established, and a basic volume level, a playback position association tag, and a limit on the number of superpositions are specified for each type of material. The scene element records involved in the display in the scene configuration data are traversed, and corresponding materials are selected from the audio material set according to the element type, and the correspondence between the materials and the scene area position is recorded, so that the spatial source direction of the sound can be derived according to the picture area subsequently. For scene elements that appear repeatedly in the same task stage, the number of occurrences and the distribution position are counted to determine the number of plays and the interval range of the sound materials in the stage. The information about the material type, the basic volume level, the spatial association relationship, and the recommended number of plays is sorted by task stage and scene area to form an audio scene structure. The output frequency and timing of the audio prompt structure and the audio scene structure are adjusted based on the interaction load coefficient to form an audio control signal; specifically, after the audio prompt structure and the audio scene structure are constructed, the interaction load coefficient is introduced to adjust the audio output rhythm. First, according to the value range of the interaction load coefficient, the current interaction state is divided into three intervals: low load, medium load, and high load. For the audio prompt structure, the time interval between adjacent prompts is shortened in the high load interval, the prompt categories closely related to the current task target are preferentially retained, and the prompts weakly related to the task target are reduced to avoid the accumulation of too many prompt events in a short time; in the low load interval, the prompt interval is appropriately lengthened, the prompt content with a slow rhythm is increased, and the prompt rhythm is matched with the relatively gentle interaction behavior of the user. For the audio scene structure, the number of background sound plays is reduced in the high load interval, the long-time continuous environmental sound is reduced, and the short scene sound associated with the key interaction event is preferentially retained; in the low load interval, the environmental sound is allowed to remain for a long time, the sound related to the scene atmosphere is increased, and the picture change and the sound change are kept in coordination. After the adjustment is completed, the playback events in the audio prompt structure and the audio scene structure are combined into a unified time axis, each audio event is arranged according to the time sequence, and a time marker, an audio material identifier, a target playback area association information, and a volume level marker are added to each playback event. If the prompt sound and the scene sound overlap in the same time slice, the preset priority rule is used to determine whether the prompt sound or the scene sound is played first, or the volume level of one type of sound is adjusted so that important prompt content can be clearly perceived. All the arranged audio event lists constitute an audio control signal, which is read and driven by an audio device to execute corresponding playback instructions in the running process.

[0033] The application further provides that the interaction detection module comprises: An initial interaction event data is formed based on the image control signal to identify the trigger position relationship between the user and the projection content; specifically, in the projection running process, the image control signal gives the position area, shape range and effective time interval of the interaction target in the current projection picture frame by frame. The interaction detection module reads these position areas in each frame, and divides the projection surface into several interactive areas and non-interactive areas. At the same time, the position point or coverage contour of the user on the projection plane in the current frame is read from the environmental perception data, and the coincidence relationship between the user position and the interactive area is compared. When the user position enters the inside of a certain interactive area, or crosses the preset boundary line from the outside of the area to enter the inside of the area, the time is marked as a trigger event, and the corresponding time, area number, interaction target type and entering method (for example, front approach or side approach) are recorded. Repeat this process to identify all trigger behaviors in consecutive frames, and accumulate the initial interaction event data in chronological order; An initial interaction trajectory data is formed based on the environmental perception data to analyze the action path information of the user in the space; specifically, on the same time axis, the user's spatial position is read in sequence from the environmental perception data, the position point of the user on the projection plane and the distance change in the depth direction are obtained. The user positions at adjacent time points are connected, and each position at a time point is regarded as a node on the trajectory, and after connecting several nodes, a continuous trajectory line is formed. When it is found that the user position crosses a preset distance threshold in the horizontal direction or the vertical direction, the position change is recorded as a key node in the trajectory, which is used to distinguish ordinary small jitter and obvious movement process. All trajectory lines in the whole interaction time are sorted out, and the start time, end time, area range and key node set of each trajectory are recorded respectively, so that the initial interaction trajectory data is formed; The sampling granularity and recording density of the initial interaction event data and the initial interaction trajectory data are adjusted based on an interaction load coefficient to form interaction event data and interaction trajectory data. Specifically, the interaction load coefficient reflects the comprehensive strength of the interaction density, the spatial trajectory change amount, and the voice trigger frequency in a unit time. To facilitate the adjustment of the sampling granularity and the recording density, the interaction load range can be divided into three intervals, i.e., low load, medium load, and high load. When processing the initial interaction event data and the initial interaction trajectory data, the interaction detection module selects different recording strategies according to the interaction load interval corresponding to the current time slice: in the high load interval, the interaction action is frequent and the trajectory changes are obvious, the sampling granularity is divided according to a short time interval, the subtle position changes and short continuous triggers are retained, and the multiple touches occurring in a short time in the same area are recorded one by one, so that the interaction event data and the interaction trajectory data are more detailed and can reflect every action change in the high-intensity interaction process; in the medium load interval, the sampling time interval is moderate, and the same type of events that repeatedly occur in a short time are processed by merging, for example, the triggers falling in the same area for multiple times are merged into a continuous trigger record, only the representative nodes are retained, and the trajectory record retains the path shape and key inflection points; in the low load interval, the sampling interval is relatively long, and the events of first entering a certain area and the obvious position change process are recorded, and the period with a longer stay time but fewer action changes is compressed, only the start and end nodes are recorded. Through the sampling and merging rules in different load intervals, the initial interaction event data and the initial interaction trajectory data are filtered and reorganized to generate interaction event data and interaction trajectory data with different levels of detail, which not only retains key interaction information but also avoids redundant records. The historical task records and the user archives are updated based on the interaction event data and the interaction trajectory data. Specifically, after the interaction event data and the interaction trajectory data are determined, the interaction events are matched to the corresponding task stages according to the task start and end times and the time ranges of each task stage recorded in the task parameter set. For each task stage, the number of interaction events, the trigger area distribution, and the typical trajectory direction in the stage are counted, and these statistical results are written into the historical task records and stored in association with the task number, the task type, and the completion status of the task. For the user archive part, the spatial areas that frequently appear in the trajectory, the common interaction methods (such as the tendency to touch with hands or kick with feet), and the activity levels under different task types are summarized and updated into the user behavior feature field. For example, the preferred operation area is marked as a high attention area, and the task type with rich interaction events is marked as a high participation task type. Through this hierarchical updating method according to the task time axis and the user dimension, traceable task interaction records are formed in the historical task records, and stable behavior feature descriptions are formed in the user archives.

[0034] It should be noted that the specific manner in which the modules and units of the learning interactive game projection system provided in the above embodiments perform operations has been described in detail in the method embodiments, and will not be described here. The learning interactive game projection system provided in the above embodiments can be divided into different functional modules to complete the above-described functions in actual application, i.e., the internal structure of the system is divided into different functional modules to complete all or part of the above-described functions, and this is not limited herein.

[0035] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be encompassed in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A learning interactive game projection system characterized by, Comprise: An environment perception module that collects user image information, spatial depth information, and voice information to form environment perception data; An interaction load construction module that forms interaction load coefficients representing interaction density, spatial trajectory change, and voice trigger frequency per unit time based on environment perception data; An identity and behavior management module that generates identity data and behavior record data based on environment perception data and historical task records; A task configuration module that generates task parameter sets and scene configuration data based on identity data, behavior record data, and interaction load coefficients; A projection control module that forms image control signals and audio control signals to drive projection devices and audio devices to work based on task parameter sets, scene configuration data, and interaction load coefficients; An interaction detection module that generates interaction event data and interaction trajectory data based on image control signals, environment perception data, and interaction load coefficients, and updates historical task records and user profiles.

2. A learning interactive game projection system according to claim 1, wherein, The environment perception module comprises: Obtain image frames containing user contour information and motion contour information in the projection coverage area to form image data; Collect spatial depth information under the time reference corresponding to the image frames to form spatial depth data; Collect voice signals during user interaction in the projection coverage area to form voice data; Correlate and organize image data, spatial depth data, and voice data according to unified time and space markers to form environment perception data.

3. A learning interactive game projection system according to claim 2, wherein, The interaction load construction module comprises: Divide the interaction behaviors of the user based on environment perception data into time segments in a continuous time interval to form an interaction behavior set corresponding to the time interval; Statistically analyze the interaction behavior set in each time interval to generate an interaction density parameter corresponding to the time interval; Organize spatial position change information in the environment perception data in the corresponding time interval to form a spatial trajectory change parameter; Statistically analyze the number of voice triggers in the environment perception data in the corresponding time interval to form a voice trigger frequency parameter; Form interaction load coefficients based on interaction density parameters, spatial trajectory change parameters, and voice trigger frequency parameters.

4. The learning interactive game projection system according to claim 1, wherein, Generating identity data comprises: Establish a target area based on facial image information in the environment perception data to form unique identity feature segments; Conduct feature consistency testing on identity feature segments in a continuous collection interval to form stable identity expression data; Compare stable identity expression data with user number information in historical task records to form identity data.

5. A learning interactive game projection system according to claim 4, wherein, Generating behavior record data comprises Extract spatial motion change information based on environment perception data in a continuous time period to form a motion segment sequence; Identify behavior change moments in the motion segment sequence to form behavior change data; Conduct time sequence association processing on behavior change data based on historical task records to form behavior record data.

6. The learning interactive game projection system according to claim 1, wherein, Generating task parameter sets comprises: Establish task matching conditions based on identity data to form task matching information; Analyze behavior persistence and behavior change location based on behavior record data to form behavior structure information; Complete task condition reorganization based on interaction load coefficients, task matching information, and behavior structure information to form task parameter sets.

7. A learning interactive game projection system according to claim 6, wherein, The generating of the scene configuration data comprises: determining a scene display interval based on the task parameter set, to form scene area data; extracting scene preference data based on the identity data and the behavior record data, to form a scene preference structure; integrating the scene area data and the scene preference structure to generate the scene configuration data.

8. The learning interactive game projection system according to claim 1, wherein, The forming of the image control signal comprises: determining an image presentation interval and task execution stage information based on the task parameter set, to form an image stage structure; extracting scene element layout relationships based on the scene configuration data, to form an image layout structure; adjusting the display rhythm of the image stage structure and the image layout structure based on the interaction load coefficient, to form the image control signal.

9. The learning interactive game projection system according to claim 1, wherein, The forming of the audio control signal comprises the following steps: extracting task prompt associated data based on the task parameter set, to form an audio prompt structure; identifying scene acoustic elements based on the scene configuration data, to form an audio scene structure; adjusting the output frequency and timing of the audio prompt structure and the audio scene structure based on the interaction load coefficient, to form the audio control signal.

10. The learning interactive game projection system according to claim 1, wherein, The interaction detection module comprises: identifying a trigger position relationship between the user and the projection content based on the image control signal, to form initial interaction event data; analyzing the action path information of the user in the space based on the environment perception data, to form initial interaction trajectory data; adjusting the sampling granularity and recording density of the initial interaction event data and the initial interaction trajectory data based on the interaction load coefficient, to form interaction event data and interaction trajectory data; updating the historical task record and the user profile based on the interaction event data and the interaction trajectory data.

Images