Music interaction method used in game, electronic equipment and storage medium

Abstract

The invention discloses a music interaction method used in games, electronic equipment and a storage medium, and relates to the field of music interaction in games, and the method comprises the following steps: establishing a character information fingerprint database in one-to-one correspondence with players; a brainwave, heart rate and electrodermal response collecting device is arranged, and physiological signals of a player in the game process are collected in real time; establishing internal and external dual modes of the game scene, and obtaining dynamic parameter information of the game scene; according to the physiological signal of the player in the game process and the dynamic parameter information of the game scene, establishing a music interaction dynamic adjustment model in the game; obtaining and recording the optimal somatosensory of music interaction in the game, and establishing an optimal somatosensory library of the music interaction in the game; and establishing a feedback evaluation system, and optimizing music interaction in the game of the player. The method has the advantages that real-time behaviors, physiological states and game scene data of players in a game are deeply analyzed, and player personalized adaptation-game scene dynamic response is generated in real time through AI.

Description

Technical Field

[0001] The present invention relates to the field of music interaction in games, and particularly to a music interaction method, an electronic device, and a storage medium for games. Background Art

[0002] Appropriate music can create a specific atmosphere for the game, enabling players to be more deeply immersed in the game world. By switching and adjusting according to different game scenarios, the game scenarios can be perfectly integrated with the game atmosphere. High-quality music can play a role in strengthening emotions and promoting the development of the plot at critical moments of the game plot, touching the emotions of players and causing resonance, thereby improving players' recognition and comfort with the game. Secondly, when players experience various plots and challenges in the game, the matching music can enhance their emotional responses, making them more engaged and enjoying the game process. Through music interaction, more interactive elements and content can be added to the game, giving players more fun in exploration and discovery.

[0003] Traditional game music interaction technologies are relatively single, usually using static background music or simple layered switching, lacking in-depth adaptation to players' real-time behaviors, physiological states, and game scenario data during the game process, resulting in a lack of interactivity between players and music. Therefore, such technologies cannot fully meet players' needs for personalized game experiences and are also difficult to create a more immersive and real-time game atmosphere. Secondly, existing technologies such as middleware like Wwise and FMOD provide dynamic music functions, but the data interaction is relatively single and cannot achieve deep integration of the game and music based on players' real-time behaviors, physiological states, and game scenario data, realizing real-time generation and adjustment of music in the game. Summary of the Invention

[0004] To solve the above technical problems, a music interaction method, an electronic device, and a storage medium for games are provided. This technical solution solves the problems raised in the above background art, namely, the lack of in-depth adaptation to players' real-time behaviors, physiological states, and game scenario data, resulting in a lack of interactivity between players and music, so that such technologies cannot fully meet players' needs for personalized game experiences and are also difficult to create a more immersive and real-time game atmosphere. Secondly, existing technologies such as middleware like Wwise and FMOD provide dynamic music functions, but the data interaction is relatively single and cannot achieve deep integration of the game and music based on players' real-time behaviors, physiological states, and game scenario data, realizing real-time generation and adjustment of music in the game.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A music interaction method for games, comprising: Extract player and in-game character information, and establish a fingerprint database of character information corresponding to each player; Set up electroencephalogram, heart rate, and skin conductance response collection devices, and wear them on the player's head and wrist respectively to collect the player's physiological signals during the game in real time; Establish an internal and external dual-mode game scenario, and obtain dynamic parameter information of the game scenario through internal program collection and external image recognition; Based on the player's physiological signals and the dynamic parameter information of the game scenario during the game, establish a dynamic adjustment model for music interaction in the game; According to the dynamic adjustment model for music interaction in the game, obtain and record the optimal physical experience of music interaction in the game, and establish an optimal physical experience library for music interaction in the game; Establish a feedback evaluation system to obtain the intuitive experience of the user's music interaction in the game, and optimize the player's music interaction in the game according to the feedback intuitive experience data.

[0006] Preferably, the establishment of the internal and external dual-mode game scenario to obtain the dynamic parameter information of the game scenario through internal program collection and external image recognition specifically includes: Extract the parameter data of the game scenario through the Unity engine or Unreal engine, and define it as the mode of game scenario parameter data collection; Set up a game process picture or streaming video collection device, set the synchronous collection frequency, and define it as the mode of game scenario parameter data collection; Among them, the synchronous collection frequency refers to that the collection frequencies of electroencephalogram, heart rate, and skin conductance response collection, and the internal and external modes of game scenario parameter data collection are kept consistent, with the same data time sequence; Among them, the internal mode data of game scenario parameter data collection includes: character speed / acceleration, character spatial position, collision force, height / gravity influence, environmental material, spatial reverberation, dynamic weather, light / shadow, particle density, number of characters; The external mode data of game scenario parameter data collection includes: character status and actions in the game, and dynamic changes in the game scenario; Perform filtering and normalization processing on the dynamic parameter information of the game scenario to eliminate the influence of peaks and dimensions in the data.

[0007] Preferably, the establishment of the dynamic loss function of the music label and the game scenario according to the corresponding relationship between the music label and the dynamic parameter information of the game scenario specifically includes: Based on the dynamic parameter information of the game scenario, establish the corresponding relationship between the sound speed and the dynamic parameter information of the game scenario; Based on the dynamic parameter information of the game scenario, establish the corresponding relationship between the change amplitude of the volume and the dynamic parameter information of the game scenario; Build the corresponding relationship between the dynamic change amount of the complexity of the timbre feature and the dynamic parameter information of the game scene according to the dynamic parameter information of the game scene; Establish the dynamic loss function of the music label and the game scene according to the corresponding relationship between each music label and the dynamic parameter information of the game scene.

[0008] Preferably, the specific steps of obtaining and recording the optimal physical experience of music interaction in the game and establishing the optimal physical experience library of music interaction in the game by dynamically adjusting the model according to the music interaction in the game include: Obtain the data after dynamic adjustment of the music label according to the dynamic adjustment model of music interaction in the game, and record and save it according to the time sequence; Obtain the physiological signal data of the player during the game with the same time sequence through the brain wave, heart rate and skin conductance response acquisition device; Filter and normalize the physiological signal data during the game process; Establish the optimal physical experience evaluation model of music interaction in the game according to the optimized physiological signal data during the game process; Obtain the optimal data of dynamic adjustment of the music label with the same time sequence according to the optimal physical experience evaluation model of music interaction in the game, and use this data as the optimal physical experience data; Establish the optimal physical experience library of music interaction in the game, and record and save the optimal physical experience data of different time sequences; The expression of the optimal physical experience evaluation model of music interaction in the game is: , In the formula, is the optimal physical experience evaluation value of music interaction in the game, , , are the cognitive and emotional evaluation weights dominated by brain waves, the physiological stress weights dominated by heart rate, and the emotional arousal weights dominated by skin conductance response respectively, is wave energy value, is wave energy value, is wave energy value, is wave energy value, reflects the degree of anxiety / tension, reflects the balance of relaxation, concentration and cognitive activation, is the weighted value of the balance of relaxation, concentration and cognitive activation, is the standard deviation of heart rate variability, is the root mean square of the difference between adjacent RR intervals, is the weight of the standard deviation of heart rate variability, is the weight of the amplitude of the skin conductance response, is the amplitude of the skin conductance response, specifically referring to the difference between the peak value generated by the skin conductance signal under a specific stimulation event and the baseline, is the value of approximate entropy, which is used to reflect the emotional stability of the player.

[0009] Preferably, the establishment of the feedback evaluation system, obtaining the intuitive experience of the user's music interaction in the game, and optimizing the player's music interaction in the game according to the feedback intuitive experience data specifically includes: Establish a feedback evaluation system, build a player music interaction experience evaluation platform, and obtain the intuitive experience of the player's music interaction in the game; According to the intuitive experience of the player's music interaction in the game, extract the characteristics of the player's physiological signals, dynamic parameter information of the game scene, and optimal somatosensory data during the game process; According to the player's intuitive experience data, based on the expert scoring method, set the importance weights for the extracted characteristics. Among them, increase the weights of the characteristics with good player feedback and decrease the weights of the characteristics with poor player feedback; Optimize the player's music interaction in the game through the corrected characteristic importance weights.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This solution provides a music interaction method for games. Through the brain wave, heart rate, and skin electrical response acquisition devices and the dual-mode inside and outside the game scene, this solution obtains the player's physiological signals and dynamic parameter information of the game scene during the game process, so as to comprehensively analyze the coordination and integration of the game and music interaction. Secondly, based on the LSTM network, a dynamic adjustment model for music interaction in the game is established. The player's physiological signals and dynamic parameter information of the game scene during the game process are used as input joint feature vectors, and music labels are output through a Softmax classifier, so as to obtain the future second's music label prediction value according to, and according to the prediction result of the model, obtain the optimal data for the dynamic adjustment of the music label with the same time sequence, and use this data as the optimal somatosensory data to establish an optimal somatosensory library for music interaction in the game, which is used to record and save the optimal somatosensory data of different time sequences. Finally, through the feedback evaluation system, obtain the intuitive experience of the user's music interaction in the game, and optimize the player's music interaction in the game according to the feedback intuitive experience data, so as to effectively deeply analyze the player's real-time behavior, physiological state, and game scene data in the game, and realize a highly personalized game music experience through AI real-time generation - player personalized adaptation - game scene dynamic response. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Flow chart of a music interaction method for a game according to the present invention; Figure 2 Flow chart of establishing a dynamic adjustment model for music interaction in a game according to physiological signals of a player and dynamic parameter information of a game scene in the present invention; Figure 3 Flow chart of establishing a dynamic loss function between a music label and a game scene according to the corresponding relationship between the music label and the dynamic parameter information of the game scene in the present invention; Figure 4 Flow chart of obtaining and recording an optimal somatosensory experience of music interaction in a game according to the dynamic adjustment model of music interaction in a game and establishing an optimal somatosensory experience library of music interaction in a game in the present invention; Figure 5 Flow chart of establishing a feedback evaluation system, obtaining an intuitive experience of a user's music interaction in a game, and optimizing the player's music interaction in a game according to the feedback intuitive experience data in the present invention; Figure 6 Structural diagram of an electronic device proposed by the present invention; Figure 7 Structural schematic diagram of a computer-readable storage medium proposed by the present invention. Detailed implementation manners

[0012] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations.

[0013] Referring to Figure 1 As shown, a music interaction method for a game includes: Extracting player and character information in the game, and establishing a character information fingerprint library corresponding to each player one by one; Setting up electroencephalogram, heart rate, and skin conductance response acquisition devices, respectively wearing them on the player's head and wrist, and collecting the player's physiological signals in real time during the game; Establishing an internal and external dual-mode game scene, and obtaining dynamic parameter information of the game scene through internal program acquisition and external image recognition methods; Establishing a dynamic adjustment model for music interaction in a game according to the player's physiological signals and the dynamic parameter information of the game scene; Obtaining and recording the optimal somatosensory experience of music interaction in a game according to the dynamic adjustment model of music interaction in a game, and establishing an optimal somatosensory experience library of music interaction in a game; Establishing a feedback evaluation system, obtaining an intuitive experience of a user's music interaction in a game, and optimizing the player's music interaction in a game according to the feedback intuitive experience data.

[0014] It can be explained that during the interaction between the game and music, it is necessary to reflect the player's gaming experience based on the player's physiological signals during the game and the dynamic parameter information of the game scene. Appropriate music can improve the player's gaming experience. Therefore, the solution obtains the player's physiological signals during the game and the dynamic parameter information of the game scene through brain wave, heart rate, and galvanic skin response acquisition devices and a dual-mode inside and outside the game scene, so as to comprehensively analyze the coordination and integration of the game and music interaction. Secondly, based on the LSTM network, a dynamic adjustment model for music interaction in the game is established. The player's physiological signals during the game and the dynamic parameter information of the game scene are used as input joint feature vectors, and music labels are output through a Softmax classifier, so as to obtain the predicted value of the music label in the future seconds. According to the prediction result of the model, obtain the optimal data for the dynamic adjustment of the music label with the same time sequence, and use this data as the optimal somatosensory data to establish an optimal somatosensory library for music interaction in the game, which is used to record and save the optimal somatosensory data of different time sequences. Finally, through the feedback evaluation system, obtain the intuitive experience of the player's music interaction in the game. According to the feedback intuitive experience data, optimize the player's music interaction in the game, so as to effectively and deeply analyze the player's real-time behavior, physiological state and game scene data in the game, and realize a highly personalized game music experience through AI real-time generation - player personalized adaptation - game scene dynamic response.

[0015] The extraction of the player and the character information in the game and the establishment of a character information fingerprint library corresponding to the player one by one specifically include: Extract the player and the character information in the game according to the player's registration information in the game; Generate the player's identity information and electronic ID according to the player and the character information in the game; Establish a character information fingerprint library corresponding to the player one by one according to the player's identity information and electronic ID, and use this character information fingerprint library as the initial character data. The character information fingerprint library includes: player gender, age, game character gender and age, character voice.

[0016] It can be explained that different players have differences in the adaptability of game and music interaction. Unified game and music interaction is difficult to meet the needs of players for personalized game experience, and it is also difficult to create a more immersive and real-time game atmosphere. Therefore, this solution establishes a character information fingerprint library corresponding to the player one by one according to the player's identity information and electronic ID, and uses this character information fingerprint library as the initial character data, providing a basic file regularization for the subsequent dynamic adjustment of the player's interaction with music after entering the game deeply.

[0017] The above-mentioned brain wave, heart rate, and skin conductance response acquisition devices are respectively worn on the player's head and wrist to collect the physiological signals of the player during the game in real time, specifically including: Set up brain wave, heart rate, and skin conductance response acquisition devices and wear them on the player's head and wrist respectively; Among them, the brain wave acquisition device is worn on the player's head to mainly collect waves, waves, waves, and waves, and obtain the , , and waveform diagrams with time series; The heart rate and skin conductance response acquisition device is worn on the player's wrist to mainly collect the player's basal heart rate, heart rate variability, skin conductance level, and skin conductance response; Adopt a synchronous acquisition method to set the acquisition frequencies of the brain wave, heart rate, and skin conductance response acquisition devices to maintain the consistency of the physiological signal data of the player during the game.

[0018] It can be explained that during the interaction between the player in the game and music, through the waves, waves, waves, and waves in the brain wave, the state of the player in the game can be effectively reflected. Among them, waves can reflect that the player is in a relaxed state, waves can reflect that the player is in a tense state, waves can reflect that the player is in a focused state. The skin conductance level can reflect the player's arousal state, and the heart rate can reflect the player's stress and recovery ability. According to the state of the brain wave, the label of the music can be adjusted to improve the comprehensive physical experience of the game and music interaction. For example, when the player's emotional balance is in a state of high arousal and excitement, the BPM can be increased (>120), and major and bright tones can be used to enhance the player's experience. Or when the player is in a state of low arousal and relaxation, a slow BPM (<80), long reverberation, and natural sound effects (running water sound) can be used to enhance the player's experience. Therefore, in this solution, by setting up brain wave, heart rate, and skin conductance response acquisition devices and adopting a synchronous acquisition method, the physiological signals of the player during the interaction between the game and music are obtained, so as to provide data support for the subsequent dynamic adjustment of music interaction in the game and obtaining the optimal physical experience of music interaction in the game.

[0019] The above-mentioned establishment of the internal and external dual modes of the game scene obtains the dynamic parameter information of the game scene through the internal program acquisition and external image recognition methods, specifically including: Extract the parameter data of the game scene through the Unity engine or Unreal engine, and define it as the mode of collecting game scene parameter data; Set up a game process picture or streaming video acquisition device, set the synchronous acquisition frequency, and define it as the mode of collecting game scene parameter data; Among them, the synchronous acquisition frequency means that the acquisition frequencies of brain waves, heart rate, skin conductance response acquisition, and the internal and external modes of game scene parameter data acquisition are kept consistent, with the same data time sequence; Among them, the internal mode data of game scene parameter data includes: character speed / acceleration, character spatial position, collision force, height / gravity influence, environmental material, spatial reverberation, dynamic weather, light / shadow, particle density, number of characters; The external mode data of game scene parameter data includes: the state and actions of characters in the game and the dynamic changes of the game scene; Perform filtering and normalization processing on the dynamic parameter information of the game scene to eliminate the influence of peaks and dimensions in the data.

[0020] It can be explained that the dynamic parameter information of the conventional game scene is mainly obtained through the Unity engine or Unreal engine, but these two methods cannot reflect the UI elements, real-world images, and the states of characters in the player's screen, so they cannot comprehensively reflect the dynamic parameter information of the game scene. Therefore, this solution establishes an internal and external dual mode for the game scene, uses image recognition technology to obtain the UI elements, real-world images, and the states of characters in the game scene, and then supplements the data through the Unity engine or Unreal engine technology to ensure the comprehensiveness of the dynamic parameter information of the game scene during the game process.

[0021] Refer to Figure 2 As shown, establishing a dynamic adjustment model for music interaction in the game according to the physiological signals of the player during the game process and the dynamic parameter information of the game scene specifically includes: Extract the principal component features in the physiological signals of the player during the game process and the dynamic parameter information of the game scene, and establish a joint feature vector; Decompose the game loading music into the dynamic change amounts of the sound speed, volume change amplitude, and timbre feature complexity, and set music tags; According to the music tags, set the edge nodes of the music change, and build the corresponding relationship between the music tags and the dynamic parameter information of the game scene through the dynamic parameter information of the game scene; According to the corresponding relationship between the music tags and the dynamic parameter information of the game scene, establish a dynamic loss function between the music tags and the game scene; Based on the LSTM network, a dynamic adjustment model for music interaction in the game is established. The joint feature vector is input, and the music label is output through the Softmax classifier to obtain the predicted value of the music label within the next seconds.

[0022] It can be explained that the physiological signals of players during the game and the dynamic parameter information of the game scene are important parameters reflecting the players' progress in the game. When dynamically adjusting the music interaction in the game, it is necessary to adjust the music label according to the physiological signals of players during the game and the dynamic parameter information of the game scene. Therefore, in this solution, by using the LSTM network, a dynamic adjustment model for music interaction in the game is established. The joint feature vector is input, and the music label is output through the Softmax classifier to obtain the predicted value of the music label within the next seconds. Among them, the input layer is used to receive the joint feature vector, the LSTM layer is used to capture the dynamic changes and patterns in the time series, the fully connected layer is used to map the output of the LSTM layer to the predicted music label or feature, and the output layer is used to output the predicted value of the music label or music feature within the next several seconds, so as to dynamically adjust the playback parameters of the music label according to the model prediction results and complete the dynamic matching between the music and the game scene.

[0023] Referring to Figure 3 as shown, the establishment of the dynamic loss function between the music label and the game scene according to the corresponding relationship between the music label and the dynamic parameter information of the game scene specifically includes: According to the dynamic parameter information of the game scene, establish the corresponding relationship between the sound speed and the dynamic parameter information of the game scene; According to the dynamic parameter information of the game scene, establish the corresponding relationship between the change range of the volume and the dynamic parameter information of the game scene; According to the dynamic parameter information of the game scene, establish the corresponding relationship between the dynamic change amount of the complexity of the timbre feature and the dynamic parameter information of the game scene; According to the corresponding relationship between each music label and the dynamic parameter information of the game scene, establish the dynamic loss function between the music label and the game scene.

[0024] It can be explained that when using the LSTM network to establish a dynamic adjustment model for music interaction in the game, it is necessary to determine the mapping relationship between the game scene and the music interaction process, and establish the loss function between the game scene and the music interaction process through the mapping relationship, so as to quantify the difference between the model prediction result and the real target through the loss function and guide the parameter optimization direction. Therefore, in this solution, by determining the mapping relationship between each music label and the game scene interaction process, a comprehensive dynamic loss function between the music label and the game scene is established to guide the parameter optimization direction; The expression of the corresponding relationship between the sound speed and the dynamic parameter information of the game scene is: , In the formula, is the dynamic change observation value of the speed of sound in the game scene, is the original state value of the speed of sound in the game scene, is a fixed sound speed value, which is 340m / s. is the relative speed of sound, where the motion towards each other is positive and the motion away from each other is negative; The corresponding expression of the volume change amplitude and the dynamic parameter information of the game scene is: , In the formula, is the dynamic change observation value of the volume change in the game scene. is the distance between the sound source and the game character, is the reference distance, generally set to 1 meter; The corresponding expression of the dynamic change of the complexity of the timbre feature and the dynamic parameter information of the game scene is: , In the formula, is the dynamic change of the complexity of the timbre characteristics in the game scene. is the complexity of the game scene, is the NPC diversity evaluation value, , , They are the weight of the comprehensive strength of the game scene complexity and NPC diversity, the weight of the game scene high-frequency events on the game process, and the weight of the dynamic change range of scene destruction, visual effect fluctuations, etc. is the trigger frequency of game scene events, is the dynamic change range of the game scene, is the sensitive value of the dynamic change index of the game scene. The larger it is, the more sensitive the game scene will be due to dynamic changes. The dynamic loss function expression of the music tag and the game scene is: , In the formula, is the dynamic loss function value of music tags and game scenes, is the equilibrium constant term, is the weight of the sound speed and the game scene dynamic loss function, is the target value when the speed of sound interacts dynamically with the game scene. is the number of samples collected, For the The weights of the tags and the dynamic loss function of the game scene, The dynamic change observations of the tags within the game scene, The target values when the tags interact dynamically with the game scene.

[0025] Referring to Figure 4 as shown, the method of dynamically adjusting the model according to music interaction in the game, obtaining and recording the optimal somatosensory experience of music interaction in the game, and establishing an optimal somatosensory experience library for music interaction in the game specifically includes: According to the model of dynamically adjusting music interaction in the game, obtaining the data of dynamically adjusted music tags, and recording and saving them according to time series; Through electroencephalogram, heart rate, and skin conductance response acquisition devices, obtaining the physiological signal data of players during the game with the same time series; Filtering and normalizing the physiological signal data during the game; According to the optimized physiological signal data during the game, establishing an optimal somatosensory experience evaluation model for music interaction in the game; According to the optimal somatosensory experience evaluation model for music interaction in the game, obtaining the optimal data of dynamically adjusted music tags with the same time series, and using this data as the optimal somatosensory data; Establishing an optimal somatosensory experience library for music interaction in the game, and recording and saving the optimal somatosensory data of different time series; The expression of the optimal somatosensory experience evaluation model for music interaction in the game is: , In the formula, is the optimal somatosensory evaluation value for music interaction in the game, , , are the weights of cognition and emotion evaluation dominated by electroencephalogram, the weight of physiological stress dominated by heart rate, and the weight of emotional arousal dominated by skin conductance response respectively, is the energy value of the wave, is the energy value of the wave, is the energy value of the wave, is to reflect the degree of anxiety / tension, is to reflect the balance of relaxation, concentration, and cognitive activation, is the weighted value of the balance of relaxation, concentration, and cognitive activation, is the standard deviation of heart rate variability, is the weight of the standard deviation of heart rate variability, is the weight of the amplitude of the skin conductance response, is the amplitude of the skin conductance response, specifically referring to the difference between the peak value and the baseline generated by the skin conductance signal under a specific stimulation event, is the value of approximate entropy, which is used to reflect the emotional stability of the player.

[0026] It can be explained that by monitoring brain waves, heart rate, and skin conductance signals, the physiological signal data of players during the game can be effectively reflected. For example, waves can reflect that the player is in a relaxed state, waves can reflect that the player is in a tense state, waves can reflect that the player is in a focused state, the skin conductance level can reflect the player's arousal state, and the heart rate can reflect the player's stress and recovery ability, etc. By observing the states of brain waves, heart rate, and skin conductance signals of players during the game, the optimal physical experience of players can be found. By recording the game and music interaction data of players at the optimal physical experience, the game and music interaction can be dynamically adjusted faster to meet the personalized game and music interaction needs of players. Therefore, this solution establishes an optimal physical experience evaluation model for music interaction in the game, comprehensively analyzes the influence of brain waves, heart rate, and skin conductance signals on players during the game and music interaction, and comprehensively evaluates the optimal physical experience of players in the game and music interaction through linear regression and weighting methods. Thus, by establishing an optimal physical experience library for music interaction in the game and supplementing the character information fingerprint library, the personalized game and music interaction needs of players are realized. Among them, the optimal physical experience library for music interaction in the game is a part of the character information fingerprint library.

[0027] Refer to Figure 5 As shown, the establishment of a feedback evaluation system, obtaining the intuitive experience of users' music interaction in the game, and optimizing the players' music interaction in the game according to the feedback intuitive experience data specifically include: Establish a feedback evaluation system, build a player music interaction experience evaluation platform, and obtain the intuitive experience of players' music interaction in the game; Extract the characteristics of the physiological signals, dynamic parameter information of the game scene, and optimal physical experience data of players during the game according to the intuitive experience of players' music interaction in the game; According to the intuitive experience data of players, based on the expert scoring method, set the attention weights for the extracted characteristics. Among them, increase the weights of the characteristics with good player feedback and reduce the weights of the characteristics with poor player feedback; Optimize the players' music interaction in the game through the corrected characteristic attention weights.

[0028] It can be explained that, to ensure the accuracy of the optimal somatosensory evaluation of the player's interaction with music in the game, this solution establishes a feedback evaluation system, builds a player music interaction experience evaluation platform, obtains the intuitive experience of the player's music interaction in the game, scores the extracted features using the expert scoring method based on the intuitive experience data feedback by the player, and sets the attention weights. Thus, by increasing the weights of the features with good player feedback and reducing the weights of the features with poor player feedback, the reliability and accuracy of the optimal somatosensory experience of the player's interaction with music in the game are further guaranteed.

[0029] Furthermore, the method according to the embodiment of the present application can also be implemented by means of Figure 6 the architecture of the electronic device shown. As Figure 6 shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to the network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, can store a method for music interaction in the game provided by the present application. The electronic device 500 may further include a user interface 508. Of course, Figure 6 the architecture shown is only exemplary. When implementing different devices, one or more components in the Figure 6 shown electronic device can be omitted according to actual needs.

[0030] Figure 7 is a schematic diagram of the structure of a computer-readable storage medium provided by an embodiment of the present application. As Figure 7 shown, it is a computer-readable storage medium 600 according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are run by a processor, a method for music interaction in the game according to the embodiment of the present application described with reference to the above drawings can be executed. The storage medium 600 includes but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0031] In summary, the advantages of the present invention are as follows: deeply analyzing the real-time behaviors, physiological states, and game scene data of players in the game, and realizing a highly personalized game music experience through AI real-time generation - player personalized adaptation - dynamic response of the game scene.

[0032] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, various changes and improvements will occur to the present invention, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection required by the present invention is defined by the appended claims and their equivalents.

Claims

1. A music interaction method for use in a game, characterized in that, Including: Extract the player and character information in the game, and establish a character information fingerprint database corresponding to each player one by one; Set up electroencephalogram (EEG), heart rate, and skin conductance response acquisition devices, and wear them on the player's head and wrist respectively to collect the player's physiological signals during the game in real time; Establish an internal and external dual-mode game scenario, and obtain the dynamic parameter information of the game scenario through internal program acquisition and external image recognition methods; Based on the player's physiological signals and the dynamic parameter information of the game scenario during the game, establish a dynamic adjustment model for music interaction in the game; According to the dynamic adjustment model for music interaction in the game, obtain and record the optimal physical experience of music interaction in the game, and establish an optimal physical experience library for music interaction in the game; Establish a feedback evaluation system, obtain the intuitive experience of the user's music interaction in the game, and optimize the player's music interaction in the game according to the feedback intuitive experience data.

2. The music interaction method for a game according to claim 1, characterized in that The extraction of the player and character information in the game and the establishment of a character information fingerprint database corresponding to each player one by one specifically include: Extract the player and character information in the game according to the player's registration information in the game; Generate the player's identity information and electronic ID according to the player and character information in the game; Establish a character information fingerprint database corresponding to each player one by one according to the player's identity information and electronic ID, and use this character information fingerprint database as the initial character data. The character information fingerprint database includes: player gender, age, game character gender and age, character voice.

3. A music interaction method for use in a game according to claim 2, characterized in that, The setting of the electroencephalogram (EEG), heart rate, and skin conductance response acquisition devices, and wearing them on the player's head and wrist respectively to collect the player's physiological signals during the game in real time specifically include: Set up electroencephalogram (EEG), heart rate, and skin conductance response acquisition devices, and wear them on the player's head and wrist respectively; Among them, the brain wave acquisition device is worn on the player's head to mainly collect the waves, waves, waves and waves, and obtain the , , and waveform diagrams with time series; Wear the heart rate and skin conductance response acquisition devices on the player's wrist to mainly collect the player's basal heart rate, heart rate variability, skin conductance level, and skin conductance response; Adopt a synchronous acquisition method, set the acquisition frequencies of the electroencephalogram (EEG), heart rate, and skin conductance response acquisition devices to maintain the consistency of the player's physiological signal data during the game.

4. A music interaction method for use in a game according to claim 3, characterized in that, The establishment of an internal and external dual-mode game scenario, and obtaining the dynamic parameter information of the game scenario through internal program acquisition and external image recognition methods specifically include: Extract the parameter data of the game scenario through the Unity engine or Unreal engine, and define it as the mode of game scenario parameter data acquisition; Set up a game process picture or streaming video acquisition device, set the synchronous acquisition frequency, and define it as the mode of game scenario parameter data acquisition; Among them, the synchronous acquisition frequency refers to that the acquisition frequencies of the electroencephalogram (EEG), heart rate, and skin conductance response acquisition, the internal mode and external mode of game scenario parameter data acquisition are kept consistent, with the same data time sequence; Among them, the internal mode data of game scenario parameter data acquisition includes: character speed / acceleration, character spatial position, collision force, height / gravity influence, environmental material, spatial reverberation, dynamic weather, light / shadow, particle density, number of characters; The external mode data of game scenario parameter data acquisition includes: character status and actions in the game, and dynamic changes in the game scenario. Filter and normalize the dynamic parameter information of the game scene to eliminate the influence of peaks and dimensions in the data.

5. The music interaction method for a game according to claim 4, wherein The establishment of the dynamic adjustment model of music interaction in the game according to the physiological signals of the player during the game and the dynamic parameter information of the game scene specifically includes: Extract the principal component features in the physiological signals of the player during the game and the dynamic parameter information of the game scene, and establish a joint feature vector; Decompose the game-loaded music into the dynamic change amounts of the sound speed, the change range of the volume, and the complexity of the timbre characteristics, and set music tags; According to the music tags, set the edge nodes of the music change, and build the corresponding relationship between the music tags and the dynamic parameter information of the game scene through the dynamic parameter information of the game scene; According to the corresponding relationship between the music tags and the dynamic parameter information of the game scene, establish the dynamic loss function between the music tags and the game scene; Based on the LSTM network, a dynamic adjustment model for music interaction in the game is established. The joint feature vector is input, and music tags are output through the Softmax classifier to obtain the predicted values of music tags in the next seconds.

6. A music interaction method for use in a game according to claim 5, characterized in that, The establishment of the dynamic loss function between the music tags and the game scene according to the corresponding relationship between the music tags and the dynamic parameter information of the game scene specifically includes: Build the corresponding relationship between the sound speed and the dynamic parameter information of the game scene according to the dynamic parameter information of the game scene; Build the corresponding relationship between the change range of the volume and the dynamic parameter information of the game scene according to the dynamic parameter information of the game scene; Build the corresponding relationship between the dynamic change amount of the complexity of the timbre characteristics and the dynamic parameter information of the game scene according to the dynamic parameter information of the game scene; According to the corresponding relationship between each music tag and the dynamic parameter information of the game scene, establish the dynamic loss function between the music tags and the game scene.

7. A music interaction method for use in a game according to claim 6, characterized in that, The obtaining and recording of the optimal body feeling of music interaction in the game according to the dynamic adjustment model of music interaction in the game, and the establishment of the optimal body feeling library of music interaction in the game specifically includes: According to the dynamic adjustment model of music interaction in the game, obtain the data after the dynamic adjustment of the music tags, and record and save them according to the time sequence; Through the brain wave, heart rate and skin conductance response acquisition device, obtain the physiological signal data of the player during the game with the same time sequence; Filter and normalize the physiological signal data during the game; According to the optimized physiological signal data during the game, establish the optimal body feeling evaluation model of music interaction in the game; According to the optimal body feeling evaluation model of music interaction in the game, obtain the optimal data of the dynamic adjustment of the music tags with the same time sequence, and use this data as the optimal body feeling data; Establish the optimal body feeling library of music interaction in the game, and record and save the optimal body feeling data of different time sequences; The expression of the optimal body feeling evaluation model of music interaction in the game is: ， In the formula, is the optimal body evaluation value of music interaction in the game, , , are respectively the cognitive and emotional evaluation weights dominated by brain waves, the physiological stress weights dominated by heart rate, and the emotional arousal weights dominated by skin conductance response, is the energy value of wave, is the energy value of wave, is the energy value of wave, is the energy value of wave, reflects the degree of anxiety / tension, reflects the balance between relaxation, concentration and cognitive activation, is the weighted value of the balance between relaxation, concentration and cognitive activation, is the standard deviation of heart rate variability, is the root mean square of the difference between adjacent RR intervals, is the weight of the standard deviation of heart rate variability, is the weight of the amplitude of skin conductance response, is the amplitude of skin conductance response, specifically referring to the difference between the peak and the baseline generated by the skin conductance signal under specific stimulation events, is the value of approximate entropy, which is used to reflect the emotional stability of the player.

8. A method for music interaction in a game according to claim 7, characterized in that The establishment of the feedback evaluation system, obtaining the intuitive experience of the player's music interaction in the game, and optimizing the player's music interaction in the game according to the feedback intuitive experience data specifically includes: Establish a feedback evaluation system, build a player music interaction experience evaluation platform, and obtain the intuitive experience of the player's music interaction in the game; According to the intuitive experience of the player's music interaction in the game, extract the features in the physiological signals of the player during the game, the dynamic parameter information of the game scene, and the optimal body feeling data; Based on the intuitive experience data of players and the expert scoring method, importance weights are set for the extracted features, among which, the weights of features with good player feedback are increased, and the weights of features with poor player feedback are decreased; Through the corrected importance weights of features, the in-game music interaction of players is optimized.

9. An electronic device, characterized in that, Including: At least one processor; And a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for in-game music interaction according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements a method for in-game music interaction according to any one of claims 1-8.

Images