Jigsaw and music combined interactive training system and method

Abstract

The invention provides a jigsaw and music combined interactive training system and method, relates to the field of education and entertainment, and optimizes the training experience of a user by dynamically adjusting the difficulty and shape of a jigsaw task and music accompaniment and combining multi-dimensional physiological and psychological information of brain wave data, emotional state and cognitive load. The system comprises a brain wave acquisition module, an emotion recognition module, a puzzle generation module, a music generation module and a sensory feedback module. The jigsaw puzzle task and the music content are adaptively adjusted according to the real-time state of the user, so that the task is not too simple and not too difficult, and emotion fluctuation or cognitive fatigue of the user is effectively avoided. Through the combination of music and tasks, the emotion state of the user can be adjusted, the concentration and cognitive competence can be improved, the learning motivation of the user can be enhanced, the method can be widely applied to the fields of mental health, cognitive training and entertainment, the cognitive competence, emotion adjustment and task completion of the user are improved, and therefore the comprehensive psychological and cognitive improvement effect is achieved.

Description

Technical Field

[0001] The present invention relates to the field of education and entertainment, and in particular to a puzzle and music combined interactive training system and method. Background Art

[0002] In the existing education and entertainment systems, most use single sensory stimulation (such as vision or hearing) and lack sufficient personalized training. Although traditional jigsaw puzzles, music training and cognitive training systems each have certain advantages, most systems fail to dynamically adjust the content according to the user's real-time status (such as attention, emotions, etc.), resulting in limited learning effects. Existing interactive training systems usually output information in a one-way manner, with low user participation and difficulty in maintaining long-term learning interest. Moreover, most systems fail to effectively integrate multiple senses such as vision, hearing, and touch, ignoring the improvement of learning efficiency and immersion through multi-sensory interaction.

[0003] Therefore, there is an urgent need for a new interactive system that can adjust the training content according to the user's real-time cognitive state and enhance the user experience and learning effect through multi-sensory stimulation. Summary of the invention

[0004] In view of this, the purpose of the present invention is to propose a puzzle and music combined interactive training system and method, which utilizes brain wave monitoring, puzzle generation and real-time music generation technology, combined with visual, auditory and tactile multi-sensory feedback, to dynamically adjust the training content and difficulty, improve learning effects, and stimulate users' interest in learning.

[0005] To achieve the above object, the present invention provides the following technical solutions: Based on the above objectives, in a first aspect, the present invention provides a puzzle and music combined interactive training system, comprising the following components: The brain wave collection module is used to collect the user's brain wave data in real time and analyze the user's cognitive load, attention and relaxation; Emotion recognition module, used to assess the user's emotional state based on their facial expressions and voice; A puzzle generation module is used to generate puzzle tasks based on the user's brain wave data and emotional state, and dynamically adjust the puzzle difficulty and puzzle shape; A music generation module, which is used to generate music style, rhythm, tone and instrument configuration that matches the puzzle task according to the user's emotional state, cognitive load, attention and relaxation; Sensory feedback module, used to provide sensory feedback data of touch, sound wave vibration and temperature sensing to enhance the user's sense of immersion; A virtual interaction module, which provides puzzle and music training in a virtual environment and supports users to interact through gestures, eye movements, and body movements; Deep learning module, which is used to analyze users’ historical behavior data and real-time performance, automatically generate personalized training recommendations, and adjust training content; A central control module is used to coordinate and adjust puzzle tasks, matching music and sensory feedback in real time.

[0006] As a further solution of the present invention, the brain wave acquisition module can monitor the user's cognitive load in real time and automatically adjust the difficulty and music style of the puzzle task according to the monitoring results.

[0007] As a further solution of the present invention, the brain wave acquisition module includes: Brainwave sensor: used to collect the user's brainwave signals in real time; Data processing unit: Analyze the collected brain wave data, extract the user's physiological indicators of cognitive load, attention, and relaxation, and generate corresponding brain wave data output.

[0008] As a further solution of the present invention, the emotion recognition module evaluates the user's emotional state through facial expression recognition, voice emotion analysis and physiological signal monitoring, and adjusts the emotional color of the music and the feedback mode of the puzzle according to different emotions.

[0009] As a further solution of the present invention, the emotion recognition module includes: Facial expression recognition device: used to capture the user's facial expression through a camera or visual sensor, and analyze the expression changes through image processing; Voice emotion analysis unit: used to analyze the user's voice emotion and detect the emotional characteristics of the voice through audio signals; Multimodal emotion recognition algorithm: Integrates facial expressions and voice data to assess the user's emotional state and output emotion labels.

[0010] As a further solution of the present invention, the puzzle generation module includes: Jigsaw puzzle generator: dynamically generates puzzle tasks based on the user’s brainwave data and emotional state; Puzzle difficulty adjustment module: adjusts the difficulty of the puzzle according to the user's cognitive load and attention indicators, including puzzle shape, quantity and complexity; Shape and Color Configurator: Adjust the shape and color of puzzle pieces based on emotion recognition results to enhance the user’s interactive experience; The jigsaw task generator automatically adjusts the composition and difficulty of the jigsaw puzzle according to the user's real-time emotional state and cognitive load, ensuring that the task is challenging but not too complex.

[0011] As a further solution of the present invention, the music generation module includes: Emotion-driven music generation unit: Generates matching music style and emotional color in real time based on the user's emotional state (such as happiness, anxiety, etc.); Cognitive load adjustment module: adjusts the rhythm, pitch and volume of music according to the user's attention and relaxation indicators to help users maintain the best state of concentration; Instrument Configurator: Dynamically adjust the use of musical instruments (such as strings, electronic sound effects, etc.) according to different training stages to provide corresponding auditory stimulation.

[0012] As a further solution of the present invention, the sensory feedback module includes: Tactile feedback device: provides users with tactile feedback through vibration and pressure to enhance the interactivity of puzzle tasks; Acoustic vibration feedback: When the user operates the puzzle, feedback is given through vibration perception to help the user perceive the completion status of the puzzle; Temperature sensing module: Provides temperature changes (such as slight heating or cooling) to enhance the user's sensory experience when the user completes a task or encounters difficulties.

[0013] As a further solution of the present invention, the virtual interaction module includes: Virtual environment creation unit: generates a virtual environment that matches the puzzle task and music generation, providing an immersive training experience; User interaction interface: supports users to interact with the virtual environment through gestures, eye movements, and body movements, and adjusts puzzle tasks and music in real time according to user actions; Motion tracking system: accurately captures the user's motion trajectory to ensure that the system responds quickly and accurately to the user's movements.

[0014] As a further solution of the present invention, the deep learning module includes: Behavioral data analysis unit: Based on the user's historical training data, analyze the user's preferences, learning curves, and emotional reactions, predict and generate personalized training recommendations; Real-time adjustment system: According to the user's performance in the current training, the puzzle difficulty, music style and sensory feedback are adjusted in real time to provide personalized training optimization solutions; Model training unit: Continuously optimize the training system through machine learning to ensure that it can adapt to the needs of different users.

[0015] As a further solution of the present invention, the central control module includes: Data coordination unit: real-time coordination of data flows between the EEG acquisition module, emotion recognition module, and puzzle generation module to ensure system synchronization; Real-time adjustment unit: automatically adjusts puzzle tasks, music content and sensory feedback based on user feedback and real-time data; Training evaluation system: Evaluates the user's overall performance and provides systematic training optimization and personalized suggestions.

[0016] In a second aspect, the present invention further provides a puzzle and music combined interactive training method, comprising the following steps: (1) Brainwave collection: collect the user's brainwave data in real time to analyze the user's cognitive load, attention, and relaxation; (2) Emotion recognition: assessing the user’s emotional state based on their facial expressions and voice; (3) Jigsaw puzzle generation: Generate puzzle tasks based on the user’s brain wave data and emotional state, and dynamically adjust puzzle difficulty and puzzle shape; (4) Music generation: Generate music style, rhythm, tone, and instrument configuration that matches the puzzle task based on the user's emotional state, cognitive load, attention, and relaxation level; (5) Sensory feedback: Provides sensory feedback data of touch, sound wave vibration and temperature sensing to enhance the user's sense of immersion; (6) Virtual interaction: Provide puzzle and music training in a virtual environment, supporting users to interact through gestures, eye movements, and body movements; (7) Deep learning: Analyze users’ historical behavior data and real-time performance, automatically generate personalized training recommendations, and adjust training content; (8) Central control: coordinate and adjust puzzle tasks, match music and sensory feedback in real time to ensure the real-time and adaptability of training.

[0017] In another aspect of the present invention, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, any one of the above-mentioned puzzle and music combination interactive training methods according to the present invention is executed.

[0018] In yet another aspect of the present invention, a computer-readable storage medium is provided, storing computer program instructions, which, when executed, implement any of the above-mentioned puzzle and music combination interactive training methods according to the present invention.

[0019] Compared with the prior art, the puzzle and music combined interactive training system and method proposed by the present invention can effectively improve the user's cognitive ability, emotion regulation, concentration and learning effect, and has the following beneficial effects: 1. The present invention can dynamically adjust the difficulty, shape and music accompaniment of the puzzle task according to the user's real-time status through real-time monitoring of physiological and psychological information including brain wave data, emotional state, cognitive load and attention. Through personalized adaptive training, it can help users train at the most suitable difficulty and emotional state, thereby improving the training effect and avoiding learning burnout or frustration caused by tasks that are too simple or too difficult.

[0020] 2. The present invention uses the emotion recognition module to regulate emotions and manage cognitive load, which can accurately assess the user's emotional state and timely adjust the difficulty of the puzzle task and the accompanying music content according to the user's emotional fluctuations. For example, when the user is nervous or anxious, the system can generate music with a slower rhythm and soft melody, and reduce the difficulty of the task to help the user regain relaxation and concentration. When the user is in a relaxed and happy state, the system can increase the challenging tasks and adjust the music rhythm to stimulate the user's cognitive potential and sense of participation. In this way, the system can effectively control the user's cognitive load and emotions, and avoid the negative effects of excessive fatigue or emotional fluctuations.

[0021] 3. The present invention uses sensory feedback modules (tactile, sonic vibration, temperature sensing, etc.) and virtual interaction modules (gestures, eye movements, body movement interactions) for immersive training experience, which can provide users with a highly immersive multi-sensory interactive experience. The timely feedback of sensory information such as tactile feedback, temperature sensing and sonic vibration can enhance the user's sense of participation and experience in the training task, thereby improving the overall training effect. In addition, the seamless combination of puzzle tasks and music content in the virtual environment can also provide users with a more immersive learning environment, enhancing the fun of training and the motivation for long-term persistence.

[0022] 4. The present invention enhances brain plasticity and cognitive training effects by continuously tracking the user's brain wave data and emotional fluctuations. It can continuously analyze the user's behavioral data and real-time performance to generate personalized training recommendations. The present invention can effectively promote brain plasticity and enhance the user's cognitive ability and concentration ability. In particular, with the close combination of jigsaw tasks and music rhythms, users can not only train their logical thinking when completing tasks, but also get sufficient sensory stimulation, thereby promoting the coordinated operation of various brain regions and the improvement of cognitive functions.

[0023] 5. The present invention can also dynamically adjust the training tasks according to the user's cognitive load and emotional state, avoiding learning burnout due to excessive fatigue or lack of challenge. At the same time, by timely adjusting the complexity of the puzzle and the emotional expression of the music, the task is more in line with the user's learning rhythm, improving the user's learning motivation and task completion. Especially when the music style matches the difficulty of the task, the system can effectively stimulate the user's emotional involvement, making the learning process more pleasant and productive.

[0024] 6. The present invention enhances interactivity and user participation through the virtual interaction module. Users can perform task operations through gestures, eye movements, body movements, etc., which not only makes puzzle training more interactive and interesting, but also enhances the intimacy and participation of users and the training system. The user's behavior and emotional feedback will be transmitted to the central control module in real time. The system can dynamically adjust the task content and training methods based on these feedbacks to ensure that each interaction can bring new challenges and a sense of accomplishment.

[0025] 7. The present invention combines music therapy and cognitive training to comprehensively improve the user's mental health and cognitive ability. By dynamically adjusting the emotional state, cognitive load and attention training content, the system can not only improve the user's cognitive ability, but also help the user manage emotions and relieve stress at the psychological level, thereby achieving the effect of mental health. The emotional regulation of music combined with the logical thinking training of the puzzle task helps to enhance the brain's ability to regulate emotions, thereby improving the user's overall mental health level.

[0026] 8. The system of the present invention can provide personalized training according to the unique needs of each user, and is suitable for users of different ages and backgrounds, especially for children, the elderly, and patients who need cognitive rehabilitation. The system of the present invention can not only be used for entertaining jigsaw puzzle games, but can also be widely used in cognitive training, emotional therapy, music therapy, psychological adjustment and other fields, and has very broad social application value.

[0027] In summary, the puzzle and music combined interactive training system and method of the present invention, with its highly personalized training method, rich sensory feedback, intelligent emotion and cognitive regulation function, and support of deep learning algorithm, can comprehensively improve the user's cognitive ability, emotional state, concentration and mental health level. Its multi-sensory, multi-module interactive design not only provides users with an immersive training experience, but also makes the training process more scientific, personalized and interesting, and ultimately achieves the goal of improving learning efficiency, stimulating emotional motivation, and optimizing cognitive function.

[0028] These and other aspects of the present application will be more concise and understandable in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the exemplary embodiments or related technical descriptions. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings: Figure 1 The present invention is a block diagram of a puzzle and music combined interactive training system according to an embodiment of the present invention.

[0030] Figure 2 The present invention is a flowchart of a method for interactive training combining puzzles and music according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Below, the present application is further described in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are intended to distinguish two non-identical entities or non-identical parameters with the same name. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, other steps or units inherent to a process, method, system, product or device that includes a series of steps or units.

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0036] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] See also Figure 1 As shown, an embodiment of the present invention provides a puzzle and music combined interactive training system, which includes a brain wave acquisition module 100, an emotion recognition module 200, a puzzle generation module 300, a music generation module 400, a sensory feedback module 500, a virtual interaction module 600, a deep learning module 700 and a central control module 800; the brain wave acquisition module 100 is used to collect the user's brain wave data in real time and analyze the user's cognitive load, attention and relaxation. Among them, the brain wave acquisition module 100 can monitor the user's cognitive load in real time, and automatically adjust the difficulty and music style of the puzzle task according to the monitoring results. The brain wave acquisition module 100 includes: a brain wave sensor: used to collect the user's brain wave signal in real time; and a data processing unit: analyze the collected brain wave data, extract the user's cognitive load, attention, and relaxation physiological indicators, and generate corresponding brain wave data output.

[0038] In this embodiment, when the brain wave acquisition module 100 is collecting, the brain wave signal (such as EEG) of the user is collected in real time through the brain wave sensor. The collected signal usually includes data of different frequency bands (such as α wave, β wave, θ wave, etc.). The data processing unit filters and extracts features from the collected brain wave data, and determines the user's cognitive load, attention and relaxation through spectrum analysis. For example: when the user is doing a jigsaw puzzle task, the brain wave signal collected by the brain wave sensor shows a higher β wave (alert state) and a lower α wave (relaxed state). The system automatically determines that the user is under a higher cognitive load and adjusts the difficulty of the jigsaw puzzle.

[0039] The present invention can dynamically adjust the difficulty, shape and music accompaniment of the puzzle task according to the user's real-time status through real-time monitoring of physiological and psychological information including brain wave data, emotional state, cognitive load and attention. Through personalized adaptive training, it can help users train at the most suitable difficulty and emotional state, thereby improving the training effect and avoiding learning burnout or frustration caused by tasks that are too simple or too difficult.

[0040] In this embodiment, the emotion recognition module 200 is used to evaluate the user's emotional state based on the user's facial expression and voice. The emotion recognition module 200 evaluates the user's emotional state through facial expression recognition, voice emotion analysis and physiological signal monitoring, and adjusts the emotional color of the music and the feedback mode of the puzzle according to different emotions.

[0041] Wherein, the emotion recognition module 200 includes: Facial expression recognition device: used to capture the user's facial expression through a camera or visual sensor, and analyze the expression changes through image processing; Voice emotion analysis unit: used to analyze the user's voice emotion and detect the emotional characteristics of the voice through audio signals; Multimodal emotion recognition algorithm: Integrates facial expressions and voice data to assess the user's emotional state and output emotion labels.

[0042] Among them, in the emotion recognition module 200, the facial expression recognition device captures the user's facial expression through a camera and uses a convolutional neural network (CNN) to analyze expression changes, such as smiling, frowning, etc. The voice emotion analysis unit analyzes the user's voice signal, uses Mel frequency cepstral coefficient (MFCC) feature extraction, and recognizes emotions through an emotion classification algorithm (such as support vector machine SVM or deep learning). The multimodal emotion recognition algorithm combines facial expressions, voice emotion analysis, and physiological signals (such as heart rate, skin electrical response) to comprehensively judge the user's emotional state. For example: when the user shows an anxious expression and voice signal, the system detects negative emotions and adjusts the music style to a soothing melody.

[0043] The present invention uses the emotion recognition module 200 to perform emotion regulation and cognitive load management, and can accurately assess the user's emotional state, and timely adjust the difficulty of the puzzle task and the accompanying music content according to the user's emotional fluctuations. For example, when the user is nervous or anxious, the system can generate music with a slower rhythm and soft melody, and reduce the difficulty of the task to help the user restore relaxation and concentration. When the user is in a relaxed and happy state, the system can increase the challenging tasks and adjust the music rhythm to stimulate the user's cognitive potential and sense of participation. In this way, the system can effectively control the user's cognitive load and emotions, and avoid the negative effects of excessive fatigue or emotional fluctuations.

[0044] In this embodiment, the puzzle generation module 300 is used to generate puzzle tasks according to the user's brain wave data and emotional state, and dynamically adjust the puzzle difficulty and puzzle shape. The puzzle generation module 300 includes: Jigsaw puzzle generator: dynamically generates puzzle tasks based on the user’s brainwave data and emotional state; Puzzle difficulty adjustment module: adjusts the difficulty of the puzzle according to the user's cognitive load and attention indicators, including puzzle shape, quantity and complexity; Shape and Color Configurator: Adjust the shape and color of puzzle pieces based on emotion recognition results to enhance the user’s interactive experience; The jigsaw task generator automatically adjusts the composition and difficulty of the jigsaw puzzle according to the user's real-time emotional state and cognitive load, ensuring that the task is challenging but not too complex.

[0045] In the puzzle generation module 300, the puzzle task generator automatically generates the shape, number and complexity of the puzzle according to the user's brain wave data and emotional state. The puzzle difficulty adjustment module adjusts the difficulty of the puzzle task according to the user's cognitive load and attention. For example, when the user's cognitive load is high, the system reduces the number of puzzle pieces and the complexity of the shape. The shape and color configurator adjusts the color and shape of the puzzle according to the emotion recognition result to match the user's emotion.

[0046] When the system recognizes that the user is in a state of high concentration, the complexity of the puzzle is increased (for example, the number of puzzle pieces and the complexity of their shapes are increased); when the system recognizes that the user is anxious, the color of the puzzle may be adjusted to warm colors and the shape may be simpler.

[0047] In this embodiment, the music generation module 400 is used to generate music style, rhythm, tone and instrument configuration that matches the jigsaw puzzle task according to the user's emotional state, cognitive load, attention and relaxation. The music generation module 400 includes: Emotion-driven music generation unit: Generates matching music style and emotional color in real time based on the user's emotional state (such as happiness, anxiety, etc.); Cognitive load adjustment module: adjusts the rhythm, pitch and volume of music according to the user's attention and relaxation indicators to help users maintain the best state of concentration; Instrument Configurator: Dynamically adjust the use of musical instruments (such as strings, electronic sound effects, etc.) according to different training stages to provide corresponding auditory stimulation.

[0048] In the music generation module 400, the emotion-driven music generation unit generates music that matches the emotional state in real time. The system selects the style and rhythm of the music by analyzing emotions (such as pleasure, anxiety). The cognitive load adjustment module adjusts the rhythm and volume of the music according to the user's attention and relaxation to maintain the best concentration state. The instrument configurator dynamically adjusts the instruments used according to the training stage (such as when the task starts and when the task is completed). For example: when the user is anxious, the system plays gentle string music and soft melodies; when the user is relaxed, add electronic music with a strong sense of rhythm.

[0049] In this embodiment, the sensory feedback module 500 is used to provide sensory feedback data of touch, sound wave vibration and temperature sensing to enhance the user's immersion. The sensory feedback module 500 includes: Tactile feedback device: provides users with tactile feedback through vibration and pressure to enhance the interactivity of puzzle tasks; Acoustic vibration feedback: When the user operates the puzzle, feedback is given through vibration perception to help the user perceive the completion status of the puzzle; Temperature sensing module: Provides temperature changes (such as slight heating or cooling) to enhance the user's sensory experience when the user completes a task or encounters difficulties.

[0050] In the sensory feedback module 500, the tactile feedback device transmits user feedback information through vibration or pressure; the acoustic vibration feedback provides vibration feedback according to the progress of the user's operation of the puzzle; the temperature sensing module feedbacks temperature changes (such as heating or cooling) to enhance the experience when the user completes the task or encounters difficulties. For example: when the user completes the puzzle, the system gives confirmation through slight vibration; when encountering difficulties, the system provides warm sensory feedback to enhance the sense of completion.

[0051] The present invention uses the sensory feedback module 500 (tactile, acoustic vibration, temperature sensing, etc.) and the virtual interaction module 600 (gesture, eye movement, body movement interaction) to provide an immersive training experience, which can provide users with a highly immersive multi-sensory interactive experience. The timely feedback of sensory information such as tactile feedback, temperature sensing and acoustic vibration can enhance the user's sense of participation and experience in the training task, thereby improving the overall training effect. In addition, the seamless combination of puzzle tasks and music content in the virtual environment can also provide users with a more immersive learning environment, enhance the fun of training and the motivation for long-term persistence.

[0052] In this embodiment, the virtual interaction module 600 is used to provide puzzle and music training in a virtual environment, and supports users to interact through gestures, eye movements and body movements. The virtual interaction module 600 includes a virtual environment creation unit, a user interaction interface and a motion tracking system. The virtual environment creation unit is used to generate a virtual environment that matches the puzzle task and music generation, providing an immersive training experience; the user interaction interface supports users to interact with the virtual environment through gestures, eye movements, and body movements, and adjusts the puzzle tasks and music in real time according to user movements; the motion tracking system is used to accurately capture the user's movement trajectory to ensure that the system responds quickly and accurately to the user's movements, for example: the user interacts with the virtual puzzle pieces by waving his arms, and the system adjusts the size and color of the puzzle pieces.

[0053] The present invention enhances interactivity and user participation through the virtual interaction module 600. Users can perform task operations through gestures, eye movements, body movements, etc., which not only makes puzzle training more interactive and interesting, but also enhances the intimacy and participation of users and the training system. The user's behavior and emotional feedback will be transmitted to the central control module 800 in real time, and the system can dynamically adjust the task content and training methods based on these feedbacks to ensure that each interaction can bring new challenges and a sense of accomplishment.

[0054] In this embodiment, the deep learning module 700 is used to analyze the user's historical behavior data and real-time performance, automatically generate personalized training recommendation content, and adjust the training content. The deep learning module 700 includes a behavior data analysis unit, a real-time adjustment system, and a model training unit. The behavior data analysis unit is used to analyze the user's preferences, learning curves, and emotional responses based on the user's historical training data, predict and generate personalized training recommendations; the real-time adjustment system is used to adjust the puzzle difficulty, music style, and sensory feedback in real time according to the user's performance in the current training, and provide a personalized training optimization plan; the model training unit continuously optimizes the training system through machine learning to ensure that it can adapt to the needs of different users.

[0055] The present invention can dynamically adjust the training tasks according to the user's cognitive load and emotional state, avoiding learning burnout due to excessive fatigue or lack of challenge. At the same time, by timely adjusting the complexity of the puzzle and the emotional expression of the music, the task is more in line with the user's learning rhythm, improving the user's learning motivation and task completion. Especially when the music style matches the task difficulty, the system can effectively stimulate the user's emotional involvement, making the learning process more pleasant and productive.

[0056] In this embodiment, the central control module 800 is used to coordinate and adjust the puzzle task, matching music and sensory feedback in real time. The central control module 800 includes a data coordination unit, a real-time adjustment unit and a training evaluation system. The data coordination unit is used to coordinate the data flow between the brain wave acquisition module 100, the emotion recognition module 200 and the puzzle generation module 300 in real time to ensure the synchronization of the system; the real-time adjustment unit is used to automatically adjust the puzzle task, music content and sensory feedback according to user feedback and real-time data; the training evaluation system is used to evaluate the overall performance of the user and provide system training optimization and personalized suggestions.

[0057] The present invention enhances brain plasticity and cognitive training effects by continuously tracking the user's brain wave data and mood fluctuations. It can continuously analyze the user's behavioral data and real-time performance to generate personalized training recommendations. The present invention can effectively promote brain plasticity and enhance the user's cognitive ability and concentration ability. In particular, with the close combination of jigsaw tasks and music rhythms, users can not only perform logical thinking training when completing tasks, but also receive sufficient sensory stimulation, thereby promoting the coordinated operation of various brain regions and the improvement of cognitive functions.

[0058] The present invention combines music therapy and cognitive training to comprehensively improve the user's mental health and cognitive ability. By dynamically adjusting the emotional state, cognitive load and attention training content, the system can not only improve the user's cognitive ability, but also help the user manage emotions and relieve stress at the psychological level, achieving the effect of mental health. The emotional regulation of music combined with the logical thinking training of the puzzle task helps to enhance the brain's ability to regulate emotions, thereby improving the user's overall mental health level.

[0059] In the second aspect of the embodiment of the present invention, see Figure 2 As shown, the present invention also provides a puzzle and music combined interactive training method, the method comprising the following steps: Step S10, brain wave collection: collect the user's brain wave data in real time to analyze the user's cognitive load, attention and relaxation; Step S20, emotion recognition: evaluating the user's emotional state based on the user's facial expression and voice; Step S30, puzzle generation: generating a puzzle task according to the user's brain wave data and emotional state, and dynamically adjusting the puzzle difficulty and puzzle shape; Step S40, music generation: generating music style, rhythm, tone and instrument configuration matching the jigsaw task according to the user's emotional state, cognitive load, attention and relaxation level; Step S50, sensory feedback: providing sensory feedback data of touch, sound wave vibration and temperature sensing to enhance the user's sense of immersion; Step S60, virtual interaction: providing puzzle and music training in a virtual environment, supporting users to interact through gestures, eye movements and body movements; Step S70, deep learning: analyzing the user's historical behavior data and real-time performance, automatically generating personalized training recommendation content, and adjusting the training content; Step S80, central control: coordinate and adjust the puzzle tasks, match the music and sensory feedback in real time to ensure the real-time and adaptability of the training.

[0060] The system of the present invention can provide personalized training according to the unique needs of each user, and is suitable for users of different ages and backgrounds, especially for children, the elderly, and patients who need cognitive rehabilitation. The system of the present invention can not only be used for entertaining jigsaw puzzle games, but can also be widely used in cognitive training, emotional therapy, music therapy, psychological adjustment and other fields, and has very broad social application value.

[0061] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0062] It should be understood that, although described in a certain order, these steps are not necessarily performed in sequence in the above order. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

[0063] In summary, the puzzle and music combined interactive training system and method of the present invention, with its highly personalized training method, rich sensory feedback, intelligent emotion and cognitive regulation function, and support of deep learning algorithm, can comprehensively improve the user's cognitive ability, emotional state, concentration and mental health level. Its multi-sensory, multi-module interactive design not only provides users with an immersive training experience, but also makes the training process more scientific, personalized and interesting, and ultimately achieves the goal of improving learning efficiency, stimulating emotional motivation, and optimizing cognitive function.

[0064] According to a third aspect of an embodiment of the present invention, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method of any one of the above embodiments is implemented.

[0065] According to a third aspect of the embodiments of the present invention, a storage medium is provided, which stores computer program instructions, and when the computer program instructions are executed, any one of the above-mentioned puzzle and music combined interactive training methods according to the present invention is implemented. It should be understood that, in the absence of conflict, all the embodiments, features and advantages described above for the puzzle and music combined interactive training method according to the present invention are also applicable to the puzzle and music combined interactive training method and storage medium according to the present invention.

[0066] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0067] Finally, it should be noted that the computer-readable storage medium (e.g., memory) herein may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. As an example and not by way of limitation, a non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. A volatile memory may include a random access memory (RAM), which may act as an external cache memory. As an example and not by way of limitation, RAM may be obtained in a variety of forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to include, but are not limited to, these and other suitable types of memory.

[0068] The various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure herein may be implemented or executed using the following components designed to perform the functions herein: a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and / or any other such configuration.

[0069] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0070] It should be understood that, as used herein, the singular form "a" or "an" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the items listed in association. The serial numbers of the embodiments disclosed in the above embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0071] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A puzzle and music combined interactive training system, characterized in that: The system consists of the following components: The brain wave collection module is used to collect the user's brain wave data in real time and analyze the user's cognitive load, attention and relaxation; Emotion recognition module, used to assess the user's emotional state based on their facial expressions and voice; A puzzle generation module is used to generate puzzle tasks based on the user's brain wave data and emotional state, and dynamically adjust the puzzle difficulty and puzzle shape; A music generation module, which is used to generate music style, rhythm, tone and instrument configuration that matches the puzzle task according to the user's emotional state, cognitive load, attention and relaxation; Sensory feedback module, used to provide sensory feedback data of touch, sound wave vibration and temperature sensing; A virtual interaction module, which provides puzzle and music training in a virtual environment and supports users to interact through gestures, eye movements, and body movements; Deep learning module, which is used to analyze users’ historical behavior data and real-time performance, automatically generate personalized training recommendations, and adjust training content; A central control module is used to coordinate and adjust puzzle tasks, matching music and sensory feedback in real time.

2. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The emotion recognition module evaluates the user's emotional state through facial expression recognition, voice emotion analysis and physiological signal monitoring, and adjusts the emotional color of the music and the feedback mode of the puzzle according to different emotions.

3. The jigsaw puzzle and music combined interactive training system as claimed in claim 2, characterized in that: The emotion recognition module comprises: Facial expression recognition device: used to capture the user's facial expression through a camera or visual sensor, and analyze the expression changes through image processing; Voice emotion analysis unit: used to analyze the user's voice emotion and detect the emotional characteristics of the voice through audio signals; Multimodal emotion recognition algorithm: Integrates facial expressions and voice data to assess the user's emotional state and output emotion labels.

4. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The puzzle generation module comprises: Jigsaw puzzle generator: dynamically generates puzzle tasks based on the user’s brainwave data and emotional state; Puzzle difficulty adjustment module: adjusts the difficulty of the puzzle according to the user's cognitive load and attention indicators, including puzzle shape, quantity and complexity; Shape and Color Configurator: adjust the shape and color of the puzzle according to the emotion recognition results; The jigsaw task generator automatically adjusts the composition and difficulty of the jigsaw puzzle according to the user's real-time emotional state and cognitive load.

5. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The music generation module comprises: Emotion-driven music generation unit: generates matching music style and emotional color in real time according to the user's emotional state; Cognitive load adjustment module: adjusts the rhythm, pitch and volume of music according to the user's attention and relaxation indicators; Instrument Configurator: Dynamically adjust the configuration of instruments according to different training stages to provide corresponding auditory stimulation.

6. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The sensory feedback module comprises: Tactile feedback device: provides users with tactile feedback through vibration and pressure; Acoustic vibration feedback: When the user operates the puzzle, feedback is given through vibration perception to help the user perceive the completion status of the puzzle; Temperature sensing module: Provides temperature changes to enhance the user's sensory experience when the user completes a task or encounters difficulties.

7. The jigsaw puzzle and music combined interactive training system as claimed in claim 6, characterized in that: The virtual interaction module includes: Virtual environment creation unit: generates a virtual environment that matches the puzzle task and music generation, providing an immersive training experience; User interaction interface: supports users to interact with the virtual environment through gestures, eye movements, and body movements, and adjusts puzzle tasks and music in real time according to user actions; Motion tracking system: accurately captures the user's motion trajectory to ensure that the system responds quickly and accurately to the user's movements.

8. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The deep learning module includes: Behavioral data analysis unit: Based on the user's historical training data, analyze the user's preferences, learning curves, and emotional reactions, predict and generate personalized training recommendations; Real-time adjustment system: According to the user's performance in the current training, the puzzle difficulty, music style and sensory feedback are adjusted in real time to provide personalized training optimization solutions; Model training unit: Continuously optimize the training system through machine learning.

9. The jigsaw puzzle and music combined interactive training system as claimed in claim 1, characterized in that: The central control module comprises: Data coordination unit: coordinates the data flow between the EEG acquisition module, emotion recognition module and puzzle generation module in real time; Real-time adjustment unit: automatically adjusts puzzle tasks, music content and sensory feedback based on user feedback and real-time data; Training evaluation system: Evaluates the user's overall performance and provides systematic training optimization and personalized suggestions.

10. A puzzle and music combined interactive training method, characterized in that: The method is performed based on the puzzle and music combined interactive training system as described in any one of claims 1 to 9, and the method comprises the following steps: Collect users’ brain wave data in real time to analyze their cognitive load, attention and relaxation; Assess the user's emotional state based on their facial expressions and voice; Generate jigsaw puzzle tasks based on the user's brain wave data and emotional state, and dynamically adjust the difficulty and shape of the puzzle; Generate music style, tempo, tone and instrument configuration that matches the puzzle task based on the user's emotional state, cognitive load, attention and relaxation; Provide sensory feedback data of touch, sound wave vibration and temperature sensing; Provide puzzle and music training in a virtual environment, supporting users to interact through gestures, eye movements, and body movements; Analyze users' historical behavior data and real-time performance, automatically generate personalized training recommendations, adjust training content, coordinate and adjust puzzle tasks, match music and sensory feedback in real time.

Images