Somatosensory dance method based on user emotion
By detecting user emotions in real time and adjusting the difficulty of somatosensory dance games, the problem of mismatch in game difficulty in the existing technology has been solved, significantly improving the user experience.
Patent Information
- Application Number
- CN202411097586.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
AI Technical Summary
The existing somatosensory dance games lack dynamic adjustments to the user's real-time emotional state in their design and operation, resulting in mismatch in game difficulty and affecting the user's gaming experience and sense of participation.
By obtaining user video data, detecting facial images, determining user's dominant emotions, and adjusting game difficulty according to emotions, using pre-trained emotion recognition model and word embedding model to calculate emotion values, adjusting dance action attributes and game difficulty, ensuring the difficulty and The user's emotional state matches.
The personalized difficulty adjustment of somatosensory dance games has been achieved, which has improved the user's sense of participation and satisfaction, and improved the overall gaming experience.
Smart Images

Figure CN118987604A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of somatosensory game technology, and in particular to a somatosensory dance method and device based on user emotions, and a computer-readable storage medium. Background Art
[0002] In existing somatosensory dance games, the game process is usually controlled by the user's body movements. Such games use cameras or sensors to capture the user's movements and compare them with preset dance movements to determine whether the user's movements meet the requirements. This interactive method allows users to obtain the dual effects of entertainment and exercise by imitating and creating dance movements in the game.
[0003] However, existing somatosensory dance games mainly rely on fixed difficulty settings and pre-arranged interaction modes during design and operation, without dynamic adjustments based on the user's real-time emotional state. This single interaction method limits the depth and personalization of the game experience.
[0004] Specifically, during the game, users may feel frustrated or bored due to inappropriate game difficulty. For example, when the game difficulty is too high, users may feel frustrated because they cannot complete the specified actions, which reduces their enthusiasm and participation in the game; when the game difficulty is too low, users may feel bored due to the lack of challenge, resulting in a monotonous and boring gaming experience. The above problems affect the overall gaming experience of users to a certain extent. Summary of the invention
[0005] The embodiment of the present application aims to achieve personalized adjustment of the game difficulty of a somatosensory dance game by providing a somatosensory dance method based on user emotions.
[0006] To achieve the above purpose, the embodiment of the present application provides a somatosensory dance method based on user emotions, comprising:
[0007] Get video data of the user during the game;
[0008] Detecting a facial image of a user from the video data;
[0009] determining the dominant emotion of the user at the current dance stage based on the facial image;
[0010] Adjusts the game difficulty of the next dance phase of the current song based on the determined dominant emotion.
[0011] In one embodiment, determining the dominant emotion of the user at the current dance stage according to the facial image includes:
[0012] Inputting the facial image into a pre-trained emotion recognition model to recognize user emotions;
[0013] Storing the user emotion recognition results in a preset emotion queue based on time sequence;
[0014] The occurrence frequency of each emotion in the emotion queue is calculated to determine the dominant emotion of the user in the current dance stage.
[0015] In one embodiment, adjusting the game difficulty of the next dance stage of the current song according to the determined dominant emotion includes:
[0016] Mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value;
[0017] Calculate dance movement adjustment parameters according to the emotion value corresponding to the dominant emotion;
[0018] The dance action attributes of the next dance stage of the current song are adjusted based on the dance action adjustment parameters.
[0019] In one embodiment, mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value includes:
[0020] Generate a word vector corresponding to the dominant emotion based on a pre-trained word embedding model;
[0021] The word vector is used as input, and the emotion value corresponding to the dominant emotion in the preset emotion parameter range is calculated based on the pre-fitted mapping function.
[0022] In one embodiment, the mapping function is a hyperbolic tangent function.
[0023] In one embodiment, before calculating the dance action adjustment parameter according to the emotion value corresponding to the dominant emotion, adjusting the game difficulty of the next dance stage of the current song according to the determined dominant emotion further includes:
[0024] It is determined whether the emotion value is within a preset interval, and if not, a dance movement adjustment parameter is calculated according to the emotion value.
[0025] In one embodiment, calculating the dance movement adjustment parameter according to the emotion value comprises at least one of the following steps:
[0026] Calculating a tolerance parameter for dance action determination according to the emotion value and a preset tolerance formula;
[0027] Calculating the time window parameters for dance action determination according to the emotion value and a preset time window formula; and
[0028] The occurrence frequency parameter of the dance action is calculated according to the emotion value and a preset frequency formula.
[0029] In one embodiment, after the current song ends, the method further includes:
[0030] Perform frequency statistics on the user's dominant emotions in all dance stages of the current song to obtain the user's comprehensive dominant emotions in the current song;
[0031] determining the type and difficulty of the next song according to the comprehensive dominant emotion;
[0032] The next song is selected from the song library according to the type and difficulty of the next song.
[0033] To achieve the above objectives, an embodiment of the present application also proposes a somatosensory dance device based on user emotions, comprising a memory, a processor, and a somatosensory dance program based on user emotions stored in the memory and executable on the processor, wherein when the processor executes the somatosensory dance program based on user emotions, the somatosensory dance method based on user emotions as described in any one of the above items is implemented.
[0034] To achieve the above objectives, an embodiment of the present application further proposes a computer-readable storage medium, on which a somatosensory dance program based on user emotions is stored. When the somatosensory dance program based on user emotions is executed by a processor, the somatosensory dance method based on user emotions as described in any of the above items is implemented.
[0035] The somatosensory dance game based on user emotions of the technical solution of the present application obtains the video data of the user's dance during the game, then monitors the user's facial image from the video data, and then determines the user's dominant emotion in the current dance stage based on these facial images, and finally adjusts the game difficulty of the next dance stage of the current song based on the dominant emotion. In this way, the difficulty can be adjusted in real time according to the user's emotions, ensuring that the game difficulty matches the user's emotional state and ability level, improving the personalized experience of the game, and then ensuring that the user maintains a high sense of participation and satisfaction during the game, thereby improving the overall game experience. It can be seen that compared with traditional somatosensory dance games, the somatosensory dance game method of the technical solution of the present application has the advantages of providing personalized game difficulty adjustment according to the user's emotions, enhancing the sense of game participation, and significantly improving the user's game experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0037] Figure 1It is a module structure diagram of an embodiment of a somatosensory dance device based on user emotions of the present invention;
[0038] Figure 2 The figure is a flow chart of an embodiment of a somatosensory dance method based on user emotions of the present invention.
[0039] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0040] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0041] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0042] It should be noted that in the claims, any reference symbols placed between brackets shall not be constructed as limitations to the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.
[0043] like Figure 1 As shown, Figure 1 It is a schematic diagram of the structure of a server 1 (also called a somatosensory dance device based on user emotions) in a hardware operating environment involved in an embodiment of the present invention.
[0044] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", smart air conditioners, smart lights, smart power supplies with networking functions, AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablet computers, e-book readers, portable computers, etc.
[0045] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12 and a network interface 13.
[0046] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc., equipped on the server 1.
[0047] Furthermore, the memory 11 may include an internal storage unit of the server 1 as well as an external storage device. The memory 11 may be used not only to store application software and various data installed on the server 1, such as the code of the somatosensory dance program 10 based on user emotions, but also to temporarily store data that has been output or is to be output.
[0048] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 11, such as executing a somatosensory dance program 10 based on user emotions.
[0049] The network interface 13 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.
[0050] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth (Blue Tooth) communication protocol or a combination thereof.
[0051] Optionally, the server may further include a user interface, which may include a display, an input unit such as a keyboard, and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be referred to as a display screen or a display unit, which is used to display information processed in the server 1 and to display a visual user interface.
[0052] Figure 1 Only the server 1 having components 11-13 and the somatosensory dance program 10 based on user emotions is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0053] In this embodiment, the processor 12 may be used to call the somatosensory dance program based on the user's emotions stored in the memory 11, and perform the following operations:
[0054] Get video data of the user during the game;
[0055] Detecting a facial image of a user from the video data;
[0056] determining the dominant emotion of the user at the current dance stage based on the facial image;
[0057] Adjusts the game difficulty of the next dance phase of the current song based on the determined dominant emotion.
[0058] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0059] Inputting the facial image into a pre-trained emotion recognition model to recognize user emotions;
[0060] Storing the user emotion recognition results in a preset emotion queue based on time sequence;
[0061] The occurrence frequency of each emotion in the emotion queue is calculated to determine the dominant emotion of the user in the current dance stage.
[0062] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0063] Mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value;
[0064] Calculate dance movement adjustment parameters according to the emotion value corresponding to the dominant emotion;
[0065] The dance action attributes of the next dance stage of the current song are adjusted based on the dance action adjustment parameters.
[0066] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0067] Generate a word vector corresponding to the dominant emotion based on a pre-trained word embedding model;
[0068] The word vector is used as input, and the emotion value corresponding to the dominant emotion in the preset emotion parameter range is calculated based on the pre-fitted mapping function.
[0069] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0070] It is determined whether the emotion value is within a preset interval, and if not, a dance movement adjustment parameter is calculated according to the emotion value.
[0071] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0072] Calculating a tolerance parameter for dance action determination according to the emotion value and a preset tolerance formula;
[0073] Calculating the time window parameters for dance action determination according to the emotion value and a preset time window formula; and
[0074] The occurrence frequency parameter of the dance action is calculated according to the emotion value and a preset frequency formula.
[0075] In one embodiment, the processor 12 may be used to call a somatosensory dance program based on user emotions stored in the memory 11, and perform the following operations:
[0076] Perform frequency statistics on the user's dominant emotions in all dance stages of the current song to obtain the user's comprehensive dominant emotions in the current song;
[0077] determining the type and difficulty of the next song according to the comprehensive dominant emotion;
[0078] The next song is selected from the song library according to the type and difficulty of the next song.
[0079] Based on the hardware architecture of the above-mentioned somatosensory dance device based on user emotions, an embodiment of the somatosensory dance method based on user emotions of the present invention is proposed. The somatosensory dance method based on user emotions of the present invention aims to realize personalized adjustment of the game difficulty of the somatosensory dance game.
[0080] Reference Figure 2 , Figure 2 This is an embodiment of the somatosensory dance method based on user emotions of the present invention, and the somatosensory dance method based on user emotions comprises the following steps:
[0081] S10: Obtain video data of the user during the game.
[0082] Among them, video data refers to the image and video information of the user during the game captured in real time by camera equipment.
[0083] Specifically, when a user starts playing a game, the user can be monitored in real time through a high-performance camera device to obtain image and video data of each frame of the user during the game. The camera device can be installed in front of the game device or in other suitable locations (such as the ceiling, wall, etc.) to ensure that the user's facial expressions are captured.
[0084] S20: Detecting a user's facial image from the video data.
[0085] Herein, the facial image refers to an image of the user's facial area extracted from video data.
[0086] Specifically, after obtaining the video data of the user's dance, in order to improve the quality of the video data, the obtained video data can be preprocessed. The preprocessing includes denoising, frame segmentation and image enhancement of the video data to improve the accuracy of subsequent face detection.
[0087] Furthermore, for the processed video data, a predefined face detection algorithm may be used to identify and locate the user's facial region from the preprocessed video data.
[0088] Optionally, any of the following face detection algorithms can be used: a Haar feature cascade classifier, a detection method combining HOG (Histogram of Oriented Gradients) features and SVM (Support Vector Machine), or a detection method based on deep learning, such as MTCNN (Multi-task Cascaded Convolutional Networks).
[0089] Based on the predefined facial detection algorithm, each frame of the video image is processed to identify the user's facial area. The facial detection algorithm determines the bounding box of the face by identifying feature points in the image, such as the location of the eyes, nose and mouth. By cropping out the identified facial area, a separate facial image can be generated.
[0090] Furthermore, to ensure the continuity and consistency of the facial image, a facial tracking algorithm is used to continuously track the face in the video frame. In each frame, the position information of the face is updated to ensure that the facial area is always accurate.
[0091] Alternatively, the face tracking algorithm may use an optical flow method, a KLT (Kanade-Lucas-Tomasi) tracking algorithm, or a deep learning based tracking method.
[0092] S30: Determine the dominant emotion of the user in the current dance stage according to the facial image.
[0093] The current dance phase refers to a specific dance action sequence or time period currently being performed during the game. Each dance phase usually consists of a set of predefined dance actions that are performed in a specific rhythm and order, and the user needs to complete these actions according to the game prompts.
[0094] Dominant emotion refers to the main emotional state displayed by the user in the current dance stage, such as happiness, anger, surprise, etc.
[0095] Specifically, when determining the dominant emotion of the user in the current dance stage, the extracted facial image needs to be preprocessed, which includes grayscale, normalization and scale transformation of the image to improve the accuracy and processing efficiency of emotion recognition.
[0096] In addition, you can use filters to remove noise from the image and enhance the contrast of the image through histogram equalization.
[0097] Furthermore, the facial key points, such as the eyes, eyebrows, nose and mouth, are located by using a predefined facial feature point detection algorithm. The position and shape information of the facial feature points are then extracted to provide feature input for emotion recognition.
[0098] Optionally, the facial landmark detection algorithm may select any of the following algorithms: Dlib's face landmark detector, OpenCV's landmark detection method, or a deep learning-based model.
[0099] Furthermore, facial feature descriptors are generated based on the detected facial feature points. These descriptors may include geometric features (such as the distance and angle between feature points) and appearance features (such as local binary pattern (LBP), histogram of directional gradients (HOG), etc.). Then, high-dimensional features are reduced in dimension through feature extraction algorithms such as PCA (principal component analysis) or LDA (linear discriminant analysis) to reduce the amount of calculation and improve the recognition effect.
[0100] Finally, based on pre-trained emotion recognition models, such as support vector machine (SVM), random forest, convolutional neural network (CNN), etc., the pre-processed facial image is input into the trained emotion recognition model, and the model classifies emotions according to the input facial features. The model outputs the emotion label corresponding to the current facial image, such as happy, angry, surprised, sad, etc., and determines the dominant emotion of the current dance stage.
[0101] It is worth noting that in order to ensure the accuracy of emotion recognition results, historical emotion data can be combined for verification, such as smoothing emotion recognition results of multiple consecutive frames to eliminate single-frame errors.
[0102] S40, adjusting the game difficulty of the next dance stage of the current song according to the identified dominant emotion.
[0103] The next dance phase refers to the sequence or time period of dance movements that will be performed after the current dance phase is completed. The design of each dance phase usually follows a certain game logic and music rhythm, so the next dance phase is a part of the current dance phase and forms a continuous dance experience with it.
[0104] The difficulty of a game is usually composed of multiple parameters, such as the tolerance of dance moves, complexity, movement speed, rhythm changes, and frequency of interactive prompts.
[0105] Specifically, after determining the dominant emotion of the user in the current dance stage, a corresponding game difficulty adjustment strategy can be formulated according to the emotion analysis result. According to the specified adjustment strategy, the game difficulty is adjusted by specifically adjusting the parameters associated with the game difficulty.
[0106] For example, when the user shows positive emotions such as happiness and excitement, the game difficulty can be appropriately increased to increase the challenge and fun.
[0107] When users show negative emotions such as anger and frustration, the game difficulty can be appropriately reduced to avoid users feeling overly challenged and frustrated.
[0108] When the user shows neutral emotions such as surprise or confusion, the current game difficulty can be kept unchanged or fine-tuned to observe the user's further reaction.
[0109] Finally, by applying the adjusted game difficulty parameters to the next dance stage of the current song, the game difficulty of the next dance stage can be adjusted.
[0110] It can be understood that by monitoring the dominant emotion of the current dance stage, the difficulty of the next stage of dance can be adjusted in real time. When entering the next stage, the difficulty of the subsequent dance stage is adjusted again by monitoring the dominant emotion of the user. In this way, the difficulty can be adjusted in real time according to the user's emotions in the somatosensory dance game, ensuring that the game difficulty matches the user's emotional state and ability level, and enhancing the personalized experience of the game. In this way, it is possible to ensure that users maintain a high sense of participation and satisfaction during the game, thereby improving the overall gaming experience.
[0111] In summary, the somatosensory dance game based on user emotions of the technical solution of the present application obtains the video data of the user's dance during the game, then monitors the user's facial image from the video data, and then determines the user's dominant emotion in the current dance stage based on these facial images, and finally adjusts the game difficulty of the next dance stage of the current song based on the dominant emotion. In this way, the difficulty can be adjusted in real time according to the user's emotions, ensuring that the game difficulty matches the user's emotional state and ability level, improving the personalized experience of the game, and then ensuring that the user maintains a high sense of participation and satisfaction during the game, thereby improving the overall game experience. It can be seen that compared with traditional somatosensory dance games, the somatosensory dance game method of the technical solution of the present application has the advantages of providing personalized game difficulty adjustment according to the user's emotions, enhancing the sense of game participation, and significantly improving the user's game experience.
[0112] In some embodiments, determining the dominant emotion of the user at the current dance stage according to the facial image includes:
[0113] S31, inputting the facial image into a pre-trained emotion recognition model to recognize user emotions.
[0114] Among them, the pre-trained emotion recognition model refers to an emotion classification model that has been trained on a large-scale facial expression database containing multiple emotion labels, such as a convolutional neural network (CNN) model, a support vector machine (SVM), a random forest, etc.
[0115] Specifically, the preprocessed facial image is used as input and fed into a pre-trained emotion recognition model, and the model outputs an emotion label, such as happy, angry, surprised, sad, etc., as the recognized user emotion.
[0116] S32: storing the user emotion recognition result into a preset emotion queue based on time sequence.
[0117] Among them, the emotion queue refers to a data structure that stores user emotion recognition results in chronological order.
[0118] Specifically, first, a preset emotion queue is initialized (established) for storing emotion recognition results. During initialization, the emotion queue is empty and is ready to start storing emotion recognition results.
[0119] The emotion queue can be a fixed-length first-in-first-out (FIFO) queue, and its length is determined by a preset time window. For example, the time window can be set to 5 seconds, and emotion recognition is performed once per second, then the queue length is 5. In this embodiment, the length of the emotion queue is determined by the duration of the current dance stage. In other words, the length of the emotion queue is consistent with the duration of the current dance stage.
[0120] Furthermore, whenever the pre-trained emotion recognition model classifies emotions on facial images and outputs emotion labels, the emotion recognition results (emotion labels) are stored in the emotion queue together with the corresponding timestamp. The timestamp is used to mark the generation time of each emotion recognition result to ensure that the data in the emotion queue is arranged in chronological order.
[0121] Among them, since the emotion queue is a first-in-first-out (FIFO) structure, new emotion recognition results will be stored at the tail of the queue. At the same time, in order to ensure that the emotion queue always contains only emotion recognition results within the preset time window, the length of the queue needs to be maintained. Specifically, when a new emotion recognition result is added to the queue, check whether the length of the queue exceeds the preset length. If the queue length exceeds the preset length, remove the earliest emotion recognition result at the head of the queue to ensure that the data in the queue is always the latest emotion recognition result. In this way, as the game progresses and emotion recognition is continuously output, the emotion queue will be continuously updated. Through this cyclic update mechanism, the emotion queue always reflects the user's emotional changes in the recent period of time.
[0122] By storing the user emotion recognition results in a preset emotion queue based on time order, it can be ensured that the data in the emotion queue is always up to date and arranged in time order, thus providing accurate data support for determining the dominant emotion.
[0123] S33. Calculate the occurrence frequency of each emotion in the emotion queue to determine the dominant emotion of the user in the current dance stage.
[0124] Among them, the frequency of occurrence refers to the ratio of the number of times each emotion label appears in the emotion queue to the total number of times.
[0125] Specifically, after obtaining the emotion queue, all stored emotion recognition results are read from the emotion queue. Then, each emotion tag in the emotion queue is counted and the number of occurrences of each emotion tag is calculated. For example, assuming that the emotion queue contains emotion tags such as happy, angry, surprised, and sad, the number of occurrences of each tag needs to be counted.
[0126] Then, based on the statistical results, the frequency of occurrence of each emotion is calculated. The calculation formula for the frequency of occurrence is:
[0127] Frequency of occurrence = total number of times in the emotion queue / number of times a certain emotion label appears.
[0128] For example, if there are 20 emotion recognition results in the emotion queue, and the happy emotion appears 8 times, then the occurrence frequency of the happy emotion is 8 / 20=0.4.
[0129] Finally, the emotion with the highest frequency is used as the dominant emotion of the user in the current dance stage. For example, if the frequency of happy emotion is the highest in the emotion queue, happy emotion is determined as the dominant emotion in the current dance stage.
[0130] It can be understood that through the above steps, the frequency of occurrence of each emotion in the emotion queue can be accurately calculated, and the dominant emotion of the user in the current dance stage can be determined. Since the emotion queue provides the complete emotion change data of the user in the current dance stage, the most representative emotional state can be obtained through frequency statistics.
[0131] In some embodiments, adjusting the game difficulty of the next dance stage of the current song according to the determined dominant emotion includes:
[0132] S41, mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value.
[0133] Among them, the emotion parameter range refers to a preset numerical range used to represent different emotion intensities.
[0134] Specifically, the system predefines a preset emotion parameter interval for representing the numerical range of different emotions. Optionally, the emotion parameter interval is a continuous numerical interval, such as [0,1], where 0 represents the lowest intensity and 1 represents the highest intensity.
[0135] After obtaining the dominant emotion of the user at the current stage, the system maps the determined dominant emotion to a preset emotion parameter range to obtain the corresponding emotion value. For example, a happy emotion can be mapped to 0.8, an angry emotion can be mapped to 0.2, and so on.
[0136] S42. Calculate dance movement adjustment parameters according to the emotion value corresponding to the dominant emotion.
[0137] Among them, dance movement adjustment parameters refer to specific parameter values used for dance movements calculated based on emotion values. The dance movement parameters include but are not limited to movement complexity (such as increasing or decreasing the number and difficulty of movements), speed (such as speeding up or slowing down the rhythm of movements), etc.
[0138] Specifically, after determining the emotional value corresponding to the dominant emotion, the system will calculate the specific dance movement adjustment parameters based on the emotional value.
[0139] Exemplarily, the dance movement adjustment parameter calculation formula may be as follows:
[0140] Action complexity = basic complexity × emotion value;
[0141] Movement speed = basic speed × emotion value.
[0142] For example, if the basic complexity is 1, the basic speed is 1, and the emotion value is 0.8, the calculated action complexity is 0.8 and the action speed is 0.8.
[0143] S43. Adjust the dance action attributes of the next dance stage of the current song based on the dance action adjustment parameters.
[0144] Among them, dance movement attributes refer to the specific characteristics of dance movements, such as complexity, speed, rhythm changes, etc.
[0145] Specifically, the dance action attributes of the next dance stage of the current song are adjusted according to the calculated dance action adjustment parameters, thereby adjusting the game difficulty of the next dance stage.
[0146] It can be understood that by mapping the dominant emotion to a specific emotion value, the user's emotional state can be quantified. The quantified emotion value can be directly involved in mathematical calculations, making the emotion analysis more objective and standardized.
[0147] In addition, by calculating the dance action adjustment parameters to adjust the dance action attributes of the next stage, and then adjusting the game difficulty of the next dance stage, the specific attributes of the dance actions, such as the complexity of the action, speed and rhythm changes, can be accurately controlled. This precise control ensures that the game difficulty can be finely adjusted according to the user's emotional state. Moreover, this method can avoid the problem of too rough or single difficulty adjustment, thereby providing a smoother and more gradual difficulty change.
[0148] In some embodiments, mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value includes:
[0149] S411. Generate a word vector corresponding to the dominant emotion based on a pre-trained word embedding model.
[0150] Among them, pre-trained word embedding models refer to models that have been trained on large-scale text data, such as Word2Vec, GloVe, or BERT, which are used to convert sentiment labels into numerical vectors.
[0151] It is worth adding that the word embedding model is a technology that represents text (words) as continuous vectors to capture the semantic and grammatical relationships between words. Through word embedding, text data can be converted into a numerical form that can be processed by machine learning models. This representation method makes words with similar meanings have similar representations in the vector space, thereby improving the effect of natural language processing tasks.
[0152] Specifically, the pre-trained word embedding model is used to convert the identified dominant emotions (such as labels such as happy, angry, surprised, sad, etc.) into corresponding word vectors. For example, the emotion "happy" can be converted into a word vector with a dimension of 300 [0.1,-0.2,0.05,...].
[0153] S412: Taking the word vector as input, and calculating the emotion value corresponding to the dominant emotion in a preset emotion parameter range based on the pre-fitted mapping function.
[0154] The pre-fitted mapping function refers to a function obtained through previous data training, which is used to map the word vector to a preset emotional parameter interval. The mapping can be achieved by a variety of methods, including simple dimension selection or complex function calculation. The pre-fitted mapping function can be a simple linear function or a nonlinear function, such as a hyperbolic tangent function (tanh).
[0155] It should be added that the output range of the hyperbolic tangent function is [-1,1], which can map the input value to a fixed interval. This feature can ensure that the emotional value is always within the preset range and avoid the occurrence of extreme values. In addition, the hyperbolic tangent function is a continuous, smooth nonlinear function with a smooth derivative near the input value. This smoothness contributes to the stability of the emotional value, ensures that the dance movement adjustment parameters do not jump frequently, and improves the consistency of the user experience. In addition, the hyperbolic tangent function is a nonlinear mapping that can compress the input value so that the output value approaches ±1 when the input value is large, and the output value is close to linear when the input value is small. This nonlinear characteristic can play a smoothing role when dealing with extreme emotions, avoiding the excessive influence of extreme emotions on the difficulty adjustment of the game.
[0156] It can be seen that by choosing the hyperbolic tangent function as the mapping function, the input sentiment word vector can be mapped to a fixed numerical range, ensuring that the output sentiment value is smooth and stable. The nonlinear mapping characteristics can better reflect the changes in sentiment intensity. Therefore, it can improve the accuracy and stability of the sentiment value, thereby improving the overall gaming experience and user satisfaction.
[0157] Furthermore, when the generated word vector is used as input, a specific dimension of the word vector or the average value of all dimensions can be selected for processing. For example, the first dimension value of the word vector can be selected, or the average value of all dimensions of the word vector can be taken.
[0158] Specifically, after the word vector obtained by converting the dominant element is input into the pre-fitted mapping function, the emotional value corresponding to the dominant emotion can be calculated.
[0159] Exemplarily, assuming that the first dimension value of the word vector is selected to be 0.1, it is input into the hyperbolic tangent function for calculation: sentiment value = tanh (0.1) = 0.0997.
[0160] Optionally, the calculated emotion values can be normalized to fit a specific value range as needed. For example, if the preset emotion parameter range is 0 to 1, the emotion value needs to be mapped from the [-1, 1] range to the [0, 1] range:
[0161] Normalized emotion value = (emotion value + 1) / 2.
[0162] It can be understood that by generating word vectors corresponding to dominant emotions based on the pre-trained word embedding model and inputting the word vectors into the pre-fitted mapping function to calculate the emotion value, the dominant emotion can be quantified into a specific value. This can improve the quantification and standardization of the emotional state and ensure the accurate mapping of the emotion value to the game difficulty adjustment parameter.
[0163] In some embodiments, before calculating the dance action adjustment parameter according to the emotion value corresponding to the dominant emotion, adjusting the game difficulty of the next dance stage of the current song according to the determined dominant emotion further includes:
[0164] It is determined whether the emotion value is within a preset interval, and if not, a dance movement adjustment parameter is calculated according to the emotion value.
[0165] The preset interval refers to a numerical range pre-set by the game developer based on the game design and user experience requirements, which is used to measure whether the emotion value is within the normal or expected range. For example, the preset interval can be set to [0.4, 0.6], indicating that the emotion value is in a "neutral" or "stable" state within this interval.
[0166] Specifically, the emotion value is compared with the upper and lower limits of the preset interval to determine whether the emotion value falls within the preset interval. For example, if the emotion value is 0.45 and the preset interval is [0.4, 0.6], the emotion value is considered to be within the preset interval.
[0167] On the contrary, if the emotion value is not within the preset range, the dance action adjustment parameters need to be calculated. At this time, the emotion value indicates that the user's emotional state may require appropriate game difficulty adjustment.
[0168] It can be understood that by determining whether the emotion value is within a preset range, the system can quickly determine whether the user's emotional state requires adjustment of the game difficulty, thereby saving system resources and improving the system's computing efficiency.
[0169] In some embodiments, calculating the dance movement adjustment parameter according to the emotion value includes at least one of the following steps:
[0170] S110, calculating a tolerance parameter for dance movement determination according to the emotion value and a preset tolerance formula.
[0171] Among them, the tolerance parameter of dance action judgment is used to adjust the requirements for the accuracy of user actions in the game, that is, the error range given when judging whether the user's action is correct. The greater the tolerance, the more lenient the game's judgment of the action; the smaller the tolerance, the stricter the judgment.
[0172] Specifically, the preset tolerance formula can be:
[0173] Tolerance parameter = β × (1-emotion value), where β is a control coefficient used to adjust the sensitivity of tolerance.
[0174] Based on the above tolerance formula, there are the following adjustment strategies:
[0175] If the emotion value is low (the user is in a bad mood), the tolerance is increased and the action accuracy requirement is lowered.
[0176] If the emotion value is high (the user is in a positive mood), the tolerance is reduced and the action accuracy requirement is increased.
[0177] S120, calculating a time window parameter for determining a dance movement according to the emotion value and a preset time window formula.
[0178] Among them, the time window parameter is used to adjust the time range of user action judgment, that is, how long it takes to complete the action is considered valid.
[0179] Specifically, the preset time window formula can be:
[0180] Time window parameter = γ × (1 + emotion value), where γ is the reference length of the time window.
[0181] Based on the above time window formula, there are the following adjustment strategies:
[0182] If the sentiment value is low, expand the time window to give users more time to complete the action.
[0183] If the emotion value is high, shorten the time window and increase the timeliness requirements of the action.
[0184] S130, calculating the occurrence frequency parameter of the dance action according to the emotion value and a preset frequency formula.
[0185] Among them, the frequency parameter is used to adjust the frequency of dance movements in the game, that is, the number of times the movement appears in a unit of time.
[0186] Specifically, the preset frequency formula can be:
[0187] Frequency parameter = δ×(1+emotion value), where δ is the reference frequency, which is used to control the basic frequency of the action.
[0188] Based on the above frequency formula, there are the following adjustment strategies:
[0189] If the emotion value is low, reduce the frequency of actions to reduce the user's operating pressure.
[0190] If the emotion value is high, the frequency of actions will be increased, increasing the challenge and the pace of the game.
[0191] It can be understood that by calculating the dance action adjustment parameters based on the emotion value, multiple key attributes of the game can be flexibly adjusted, such as the tolerance, time window and frequency of action judgment. This multi-dimensional adjustment mechanism ensures that the game difficulty matches the user's emotional state, thereby improving the game's adaptability and personalized experience.
[0192] In some embodiments, after the current song ends, the method further includes:
[0193] S210: Perform frequency statistics on the dominant emotions of the user in all dance stages of the current song to obtain the comprehensive dominant emotion of the user in the current song.
[0194] Specifically, after the current song ends, the dominant emotion data of the user in all dance stages of the current song is collected, and the frequency of each emotion is calculated. Then, based on the statistical results, the emotion with the highest frequency is determined as the user's comprehensive dominant emotion in the current song. For example, if the frequency of happy emotion is the highest in all stages, the comprehensive dominant emotion is happy.
[0195] S220: Determine the type and difficulty of the next song according to the comprehensive dominant emotion.
[0196] Specifically, the appropriate song type is selected for the user based on the overall dominant emotion. For example, if the user's overall dominant emotion is happy, a cheerful and dynamic song type can be selected; if the emotion is calm, a soothing and gentle song type can be selected.
[0197] At the same time, the difficulty of the next song is adjusted according to the overall dominant emotion. For example, if the overall emotion is positive (such as happy, excited), the difficulty can be appropriately increased; if the emotion is relatively low (such as sad, angry), a song with a lower difficulty can be selected.
[0198] S230, selecting the next song from the song library according to the type and difficulty of the next song.
[0199] Specifically, after determining the difficulty and type of the next song, suitable songs are selected from the pre-built song library according to the selected type and difficulty. Each song in the song library is marked with its type, rhythm and difficulty information for quick retrieval.
[0200] Alternatively, a recommendation algorithm (such as rule-based recommendation, collaborative filtering, etc.) can be used to select the most appropriate song based on the user's historical preferences and current comprehensive emotions. This ensures that the recommended songs not only match the user's emotions, but also provide appropriate challenges and fun.
[0201] It can be understood that by counting the frequency of the user's dominant emotions in all dance stages of the current song and determining the comprehensive dominant emotions, a scientific basis can be provided for selecting the type and difficulty of the next song. Therefore, the game content of the next song can be dynamically adjusted according to the user's emotional state, which helps to further improve the personalization and satisfaction of the user experience.
[0202] In addition, the embodiment of the present invention further proposes a computer-readable storage medium, which may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a somatosensory dance program 10 based on user emotions. The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the somatosensory dance method based on user emotions and the server 1, and will not be repeated here.
[0203] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0204] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0207] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0208] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A somatosensory dance method based on user emotions, characterized in that: include: Get video data of the user during the game; Detecting a facial image of a user from the video data; determining the dominant emotion of the user at the current dance stage based on the facial image; Adjusts the game difficulty of the next dance phase of the current song based on the determined dominant emotion.
2. The method for somatosensory dancing based on user emotions as claimed in claim 1, characterized in that: Determining the dominant emotion of the user at the current dance stage according to the facial image includes: Inputting the facial image into a pre-trained emotion recognition model to recognize user emotions; Storing the user emotion recognition results in a preset emotion queue based on time sequence; The occurrence frequency of each emotion in the emotion queue is calculated to determine the dominant emotion of the user in the current dance stage.
3. The method for somatosensory dancing based on user emotions as claimed in claim 1, characterized in that: Adjusts the game difficulty of the next dance phase of the current song based on the determined dominant emotion, including: Mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value; Calculate dance movement adjustment parameters according to the emotion value corresponding to the dominant emotion; The dance action attributes of the next dance stage of the current song are adjusted based on the dance action adjustment parameters.
4. The method for somatosensory dancing based on user emotions as claimed in claim 3, characterized in that: Mapping the dominant emotion to a preset emotion parameter interval to obtain a corresponding emotion value includes: Generate a word vector corresponding to the dominant emotion based on a pre-trained word embedding model; The word vector is used as input, and the emotion value corresponding to the dominant emotion in the preset emotion parameter range is calculated based on the pre-fitted mapping function.
5. The method for somatosensory dancing based on user emotions as claimed in claim 4, characterized in that: The mapping function is a hyperbolic tangent function.
6. The method for somatosensory dancing based on user emotions as claimed in claim 3, characterized in that: Before calculating the dance action adjustment parameter according to the emotion value corresponding to the dominant emotion, adjusting the game difficulty of the next dance stage of the current song according to the determined dominant emotion also includes: It is determined whether the emotion value is within a preset interval, and if not, a dance movement adjustment parameter is calculated according to the emotion value.
7. The method for somatosensory dancing based on user emotions as claimed in claim 3 or 6, characterized in that: Calculating dance movement adjustment parameters according to the emotion value includes at least one of the following steps: Calculating a tolerance parameter for dance action determination according to the emotion value and a preset tolerance formula; Calculate the time window parameter for dance action determination according to the emotion value and a preset time window formula; as well as The occurrence frequency parameter of the dance action is calculated according to the emotion value and a preset frequency formula.
8. The method of somatosensory dancing based on user emotions as claimed in claim 1, characterized in that: After the current song ends, the method further includes: Perform frequency statistics on the user's dominant emotions in all dance stages of the current song to obtain the user's comprehensive dominant emotions in the current song; determining the type and difficulty of the next song according to the comprehensive dominant emotion; The next song is selected from the song library according to the type and difficulty of the next song.
9. A somatosensory dance device based on user emotions, characterized in that: The invention comprises a memory, a processor and a somatosensory dance program based on user emotions which is stored in the memory and can be run on the processor. When the processor executes the somatosensory dance program based on user emotions, the somatosensory dance method based on user emotions as described in any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a somatosensory dance program based on user emotions, and when the somatosensory dance program based on user emotions is executed by a processor, the somatosensory dance method based on user emotions as described in any one of claims 1-8 is implemented.
Citation Information
Cited By
Dance video generation method, device and equipment and readable storage medium
CN121037644A
