A somatosensory interactive immersive experience room system

Abstract

The present invention discloses a somatosensory interactive immersive experience room system, comprising a first determination module for receiving a user input demand keyword and displaying recommended content according to the demand keyword; a playback module for receiving and playing the target content selected by the user from the recommended content; an identification module for identifying the scene of the target content and determining the target environmental parameters and target somatosensory information; an acquisition module for acquiring the environmental parameters and somatosensory information of the current environment; and an adjustment module for adjusting the environment according to the target environmental parameters and environmental parameters, and adjusting the somatosensory according to the target somatosensory information and somatosensory information. The somatosensory interactive immersive experience is provided to the user according to the user's needs, and the scene is accurately and comprehensively constructed, thereby improving the user experience. At the same time, based on the naked-eye 3D technology, the inconvenience of wearing 3D glasses is avoided, and the comfort and convenience are increased.

Description

Technical Field

[0001] The present invention relates to the technical field of private rooms, and in particular to a somatosensory interactive immersive experience private room system. Background Art

[0002] At present, users can only experience the scene by wearing 3D glasses, which is inconvenient and cannot fully immerse themselves in the experience. It is impossible to provide users with a somatosensory interactive immersive experience according to their needs, and the lack of environmental construction affects the user experience. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technologies to a certain extent. To this end, the purpose of the present invention is to propose a somatosensory interactive immersive experience room system, which provides users with somatosensory interactive immersive experience according to their needs, realizes accurate and comprehensive construction of scenes, improves user experience, and at the same time avoids the inconvenience of wearing 3D glasses based on naked-eye 3D technology, thereby increasing comfort and convenience.

[0004] To achieve the above purpose, the embodiment of the present invention proposes a somatosensory interactive immersive experience room system, including:

[0005] The first determination module is used to receive the demand keywords input by the user and display the recommended content according to the demand keywords;

[0006] A playback module, used to receive and play target content selected by a user from the recommended content;

[0007] The recognition module is used to recognize the scene of the target content and determine the target environment parameters and target body sensory information;

[0008] The acquisition module is used to obtain the environmental parameters and somatosensory information of the current environment;

[0009] The adjustment module is used to perform environmental adjustment according to the target environmental parameters and environmental parameters, and to perform somatosensory adjustment according to the target somatosensory information and somatosensory information.

[0010] According to some embodiments of the present invention, the first determining module includes:

[0011] The first building module is used to build a recommendation system; the recommendation system includes a title, description, preset keywords and recommendation level;

[0012] The matching module is used to receive the demand keywords input by the user, match the demand keywords with preset keywords, and determine the corresponding content as the recommended content according to the matching results.

[0013] According to some embodiments of the present invention, the playback module includes:

[0014] Glasses-free 3D surround screen;

[0015] A determination module, used to determine whether the number of target contents selected by the user from the recommended contents is 2, and when it is determined that the number is 2, generate instruction information for dividing the page of the naked-eye 3D surround screen;

[0016] The second determination module is used for:

[0017] Determine the first target content and the second target content included in the selected target content; obtain first element information of the first target content and second element information of the second target content; calculate the element difference value between the first target content and the second target content according to the first element information and the second element information;

[0018] The element difference value is compared with the preset segmentation threshold. When it is determined that the element difference value is less than the preset segmentation threshold, the naked-eye 3D surround screen is equally divided; otherwise, the preset difference value-division strategy data table is queried according to the element difference value to determine the corresponding division strategy, and the naked-eye 3D surround screen is divided according to the division strategy;

[0019] The target content is played according to the division result of the naked-eye 3D surround screen.

[0020] According to some embodiments of the present invention, the identification module includes:

[0021] The first identification module is used for:

[0022] Perform image recognition on the target content and determine the target scene;

[0023] Determine the corresponding target temperature and humidity information and target odor according to the target scene; determine the target environmental parameters according to the target temperature and humidity information and target odor;

[0024] The second identification module is used for:

[0025] Performing audio recognition on the target content to obtain audio information;

[0026] The target somatosensory information is determined by querying a preset audio-somatosensory data table according to the audio information.

[0027] According to some embodiments of the present invention, the first identification module includes:

[0028] The third determination module is used to:

[0029] Annotate element boxes in the image of the target content based on a preset trained element recognition model;

[0030] Perform convolution processing on each element frame to determine the overall convolution feature of the element frame; perform pooling processing on the overall convolution feature, perform multiple levels of residual processing on the overall convolution feature after pooling, and then perform pooling processing again to obtain the overall feature of the element frame; segment the element frame according to the overall feature, determine a number of local features, use the several local features as the feature information corresponding to each element frame, input the feature information corresponding to each element frame into the fully connected network in the element recognition model for channel connection processing, and obtain a feature map;

[0031] The fourth determination module is used for:

[0032] Determine the target element corresponding to each element box according to the feature map;

[0033] Determine the association relationship between target elements; the association relationship includes position relationship and combination relationship;

[0034] Determine the weight coefficient of each element according to the association relationship between the target elements;

[0035] Determine the target scene based on the target element and the corresponding weight coefficient.

[0036] According to some embodiments of the present invention, the fourth determining module determines the association relationship between the target elements, including:

[0037] A generation module is used to obtain page element data, scene business marketing data and scene space data; split the page element data, scene business marketing data and scene space data to obtain the corresponding component structure, and generate the corresponding JSON data, drag it to the interface editing area, and generate the target scene general map;

[0038] The activation module is used to match the target element with the elements in the target scene overall map, activate the pixel points of the corresponding elements in the target scene overall map according to the matching results, and determine the association relationship between the target elements based on the overall association relationship of the target scene overall map and the pixel points of the corresponding elements.

[0039] According to some embodiments of the present invention, the second identification module includes:

[0040] The second building module is used to build an audio recognition model;

[0041] The fifth determination module is used to perform audio recognition on the target content, obtain a number of audio clips, determine the frequency spectrum characteristics of each audio clip; input the frequency spectrum characteristics into the audio recognition model, and output audio information.

[0042] According to some embodiments of the present invention, the second building block includes:

[0043] The labeling module is used to obtain the sample spectrum features corresponding to the sample audio clips, build an audio feature training set, and label each audio clip with an audio label;

[0044] The training module is used to build an audio recognition model. The audio feature training set is input into the audio recognition model for training. Based on the difference between the output result and the annotated audio label, the model parameters of the audio recognition model are continuously adjusted until the output result is consistent with the annotated audio label, indicating that the training is completed and a trained audio recognition model is obtained.

[0045] According to some embodiments of the present invention, the adjustment module is further used to obtain motion information of a human body in the private room, and perform corresponding adjustment actions according to the motion information of the human body;

[0046] The adjustment module comprises:

[0047] The segmentation module is used to obtain the actual image and the standard image of the private room, and perform image segmentation respectively to obtain a plurality of sub-actual images and a plurality of sub-standard images;

[0048] Matching the sub-actual image with the corresponding sub-standard image, determining the unmatched sub-actual image, and extracting the contour to obtain contour features; matching the contour features with preset human body contour features, and determining the human body image according to the matching results;

[0049] a sixth determination module, configured to obtain key points of a human body according to the human body image, calculate limb vector features of each joint according to the key points of the human body, and determine motion information of the human body according to the limb vector features;

[0050] The execution module is used to execute corresponding adjustment actions according to the action information of the human body.

[0051] According to some embodiments of the present invention, the acquisition module includes:

[0052] The first acquisition module is used to:

[0053] Based on the temperature and humidity sensors installed in multiple locations in the private room, the temperature and humidity information is obtained;

[0054] Acquire odor information based on odor sensors installed in multiple locations in the private room;

[0055] Determine the environmental parameters of the current environment based on temperature, humidity and odor information;

[0056] The second acquisition module is used to acquire audio information of the current environment as somatosensory information based on audio sensors set at multiple locations in the private room.

[0057] The present invention proposes a somatosensory interactive immersive experience room system, which provides users with somatosensory interactive immersive experience according to their needs, realizes accurate and comprehensive construction of scenes, improves user experience, and avoids the inconvenience of wearing 3D glasses based on naked-eye 3D technology, thereby increasing comfort and convenience. The somatosensory interactive experience in the room is suitable for family gatherings, friends gatherings or special events, which increases the fun and interactivity of the event and allows travel without leaving home, which has many application scenarios.

[0058] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 is a block diagram of a somatosensory interactive immersive experience room system according to an embodiment of the present invention;

[0062] Figure 2 is a block diagram of a first determination module according to an embodiment of the present invention;

[0063] Figure 3 is a block diagram of a playback module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] like Figure 1 As shown, the embodiment of the present invention proposes a somatosensory interactive immersive experience room system, including:

[0066] The first determination module is used to receive the demand keywords input by the user and display the recommended content according to the demand keywords;

[0067] A playback module, used to receive and play target content selected by a user from the recommended content;

[0068] The recognition module is used to recognize the scene of the target content and determine the target environment parameters and target body sensory information;

[0069] The acquisition module is used to obtain the environmental parameters and somatosensory information of the current environment;

[0070] The adjustment module is used to perform environmental adjustment according to the target environmental parameters and environmental parameters, and to perform somatosensory adjustment according to the target somatosensory information and somatosensory information.

[0071] The working principle of the above technical solution: In this embodiment, the demand keywords include forest, ocean, canyon, etc.

[0072] In this embodiment, corresponding recommended content is given according to keywords, target content selected by the user from the recommended content is received and played, the scene of the target content is identified, and target environmental parameters are determined, including temperature, humidity and smell information in the target content. Target somatosensory information is determined, including audio information in the target content.

[0073] In this embodiment, when the environment is adjusted according to the target environment parameters and the environment parameters, the adjustment is performed based on the temperature and humidity controller and the odor generator. The odor generator provides odors that match the virtual scene, such as the fragrance of flowers, sea breeze or tropical rain forest, to enhance immersion and realism.

[0074] In this embodiment, the somatosensory adjustment is performed according to the target somatosensory information and the somatosensory information. The somatosensory adjustment is performed based on the somatosensory chair. The corresponding somatosensory adjustment is performed according to the different audio information to provide the corresponding rhythm.

[0075] The beneficial effects of the above technical solution are: giving users a somatosensory interactive immersive experience according to their needs, achieving accurate and comprehensive construction of scenes, improving user experience, and avoiding the inconvenience of wearing 3D glasses based on naked-eye 3D technology, increasing comfort and convenience. The somatosensory interactive experience in a private room is suitable for family gatherings, friends gatherings or special events, adding fun and interactivity to the event, and allowing people to travel without leaving home, which is very applicable.

[0076] like Figure 2 As shown, according to some embodiments of the present invention, the first determining module includes:

[0077] The first building module is used to build a recommendation system; the recommendation system includes a title, description, preset keywords and recommendation level;

[0078] The matching module is used to receive the demand keywords input by the user, match the demand keywords with preset keywords, and determine the corresponding content as the recommended content according to the matching results.

[0079] The working principle and beneficial effects of the above technical solution are as follows: A data structure for storing recommended content is defined to build a recommendation system. The required keywords input by the user are received, and the required keywords are matched with preset keywords, and the corresponding content is determined as the recommended content according to the matching results. Fuzzy matching algorithms (such as Levenshtein distance) are used to improve keyword matching, so as to accurately determine the recommended content.

[0080] like Figure 3 As shown, according to some embodiments of the present invention, the playback module includes:

[0081] Glasses-free 3D surround screen;

[0082] A determination module, used to determine whether the number of target contents selected by the user from the recommended contents is 2, and when it is determined that the number is 2, generate instruction information for dividing the page of the naked-eye 3D surround screen;

[0083] The second determination module is used for:

[0084] Determine the first target content and the second target content included in the selected target content; obtain first element information of the first target content and second element information of the second target content; calculate the element difference value between the first target content and the second target content according to the first element information and the second element information;

[0085] The element difference value is compared with the preset segmentation threshold. When it is determined that the element difference value is less than the preset segmentation threshold, the naked-eye 3D surround screen is equally divided; otherwise, the preset difference value-division strategy data table is queried according to the element difference value to determine the corresponding division strategy, and the naked-eye 3D surround screen is divided according to the division strategy;

[0086] The target content is played according to the division result of the naked-eye 3D surround screen.

[0087] The working principle and beneficial effects of the above technical solution: In this embodiment, when different users want to experience different target contents, the naked eye 3D surround screen is divided. Determine the first target content and the second target content included in the selected target content; obtain the first element information of the first target content and the second element information of the second target content; the first element information represents the constituent element information included in the first target content, such as the included objects, etc. The second element information represents the constituent element information included in the second target content. Calculate the element difference value between the first target content and the second target content according to the first element information and the second element information, indicating the information difference between the meaning of the first target content and the second target content, compare the element difference value with the preset segmentation threshold, and divide the naked eye 3D surround screen into equal parts when it is determined that the element difference value is less than the preset segmentation threshold; otherwise, query the preset difference value-division strategy data table according to the element difference value, determine the corresponding division strategy, divide the naked eye 3D surround screen according to the division strategy; play the target content according to the division result of the naked eye 3D surround screen. Realize the reasonable division of the naked eye 3D surround screen according to the characteristics of the content selected by the user, so that the user can watch it well and improve the user experience.

[0088] According to some embodiments of the present invention, the identification module includes:

[0089] The first identification module is used for:

[0090] Perform image recognition on the target content and determine the target scene;

[0091] Determine the corresponding target temperature and humidity information and target odor according to the target scene; determine the target environmental parameters according to the target temperature and humidity information and target odor;

[0092] The second identification module is used for:

[0093] Performing audio recognition on the target content to obtain audio information;

[0094] The target somatosensory information is determined by querying a preset audio-somatosensory data table according to the audio information.

[0095] The working principle and beneficial effects of the above technical solution are: image recognition and audio recognition are performed on the target content to accurately obtain the target environment parameters and target body sensory information.

[0096] According to some embodiments of the present invention, the first identification module includes:

[0097] The third determination module is used to:

[0098] Annotate element boxes in the image of the target content based on a preset trained element recognition model;

[0099] Perform convolution processing on each element frame to determine the overall convolution feature of the element frame; perform pooling processing on the overall convolution feature, perform multiple levels of residual processing on the overall convolution feature after pooling, and then perform pooling processing again to obtain the overall feature of the element frame; segment the element frame according to the overall feature, determine a number of local features, use the several local features as the feature information corresponding to each element frame, input the feature information corresponding to each element frame into the fully connected network in the element recognition model for channel connection processing, and obtain a feature map;

[0100] The fourth determination module is used for:

[0101] Determine the target element corresponding to each element box according to the feature map;

[0102] Determine the association relationship between target elements; the association relationship includes position relationship and combination relationship;

[0103] Determine the weight coefficient of each element according to the association relationship between the target elements;

[0104] Determine the target scene based on the target element and the corresponding weight coefficient.

[0105] The working principle and beneficial effects of the above technical solution are as follows: First, a preset trained element recognition model (such as a target detection model based on deep learning, such as YOLO, SSD, Faster R-CNN, etc.) is used to automatically mark the bounding boxes of the elements of interest (such as people, objects, scenes, etc.) in the image of the target content. Output: A series of images with element boxes are obtained, each of which identifies a specific element in the image. Convolution processing: Convolution operations are performed on the images in each element box to extract local features. The convolution layer captures local features in the image by multiplying and summing the sliding window and the weight matrix (convolution kernel). Pooling processing: Pooling (Pooling) is performed on the feature map after convolution, such as Max Pooling or Average Pooling, to reduce the dimension and calculation amount of the feature map while retaining important features. Residual processing: Multiple levels of residual processing (such as residual blocks in ResNet) are performed, which helps to solve the gradient disappearance or gradient explosion problems in deep network training and improve the feature extraction ability of the network. Re-pooling: Perform another pooling on the feature map after residual processing to further compress the feature map and prepare to enter the fully connected layer. Segmentation and local features: Segment the processed feature map according to the boundaries of the element box and extract the local features corresponding to each element box. Fully connected network: Input the local features of each element box into the fully connected network (FCN), and fuse these features through channel connection (concatenation) or summation (summation) to form a higher-level feature map. Target element recognition: According to the feature map, use a classifier (such as the Softmax layer) to determine the specific target element category corresponding to each element box. Determine the association relationship: Determine the association between target elements by analyzing the position relationship (such as adjacent, overlapping, distance, etc.) and combination relationship (such as common object combination patterns) between element boxes. Determine the weight coefficient: Assign a weight coefficient to each element based on factors such as the importance, position, size, and association of the element.

[0106] Target scene determination: Combine the target elements and their weight coefficients and use scene analysis algorithms (such as image segmentation, scene graph construction, etc.) to determine the target scene described by the entire image, so as to accurately determine the target scene.

[0107] According to some embodiments of the present invention, the fourth determining module determines the association relationship between the target elements, including:

[0108] A generation module is used to obtain page element data, scene business marketing data and scene space data; split the page element data, scene business marketing data and scene space data to obtain the corresponding component structure, and generate the corresponding JSON data, drag it to the interface editing area, and generate the target scene general map;

[0109] The activation module is used to match the target element with the elements in the target scene overall map, activate the pixel points of the corresponding elements in the target scene overall map according to the matching results, and determine the association relationship between the target elements based on the overall association relationship of the target scene overall map and the pixel points of the corresponding elements.

[0110] Working principle and beneficial effects of the above technical solution: The generation module obtains page element data, scene business marketing data and scene space data. These data are split and their composition structure is parsed. The corresponding JSON data is generated, which will describe the attributes, position, size, association, etc. of each element. The elements represented by these JSON data are dragged to the interface editing area to build the target scene map in a visual way. Data acquisition: The required data is obtained through API calls, database queries or file reading. Data parsing: Write a parser or use existing libraries (such as JSON.parse, XMLParser, etc.) to parse the obtained data and decompose it into operable components or objects. JSON generation: Generate a JSON object for each element, containing all its necessary information (such as ID, type, position, size, style, data binding, etc.). Interface integration: Develop a front-end interface editing area and use technologies such as JavaScript, HTML5 and CSS to dynamically render JSON data into visual elements. Drag-and-drop libraries (such as SortableJS, Vue.Draggable, etc.) can be used to enhance the user experience. Scene construction: Users build the target scene map in the interface editing area by dragging and dropping. The activation module matches the target element with the elements in the target scene map. Based on the matching results, the pixels of the corresponding elements in the target scene map are activated. The associations between the target elements are determined, and the dynamic behavior or visual effects of the scene are adjusted based on these relationships. Element matching: Implement a matching algorithm to compare the target element (which may be specified by ID, type, or other unique identifier) ​​with the elements in the target scene map. Pixel activation: Based on the matching results, the pixels of the corresponding element are "activated" by modifying the CSS style (such as border color, background color change, animation effect, etc.) on the front-end interface. This involves DOM operations and dynamic application of CSS styles. Association determination: Based on the overall layout of the target scene map and the logical relationships between elements (such as parent-child relationships, adjacent relationships, dependency relationships, etc.), the association relationships between the target elements are determined. This requires analyzing specific fields in the JSON data or executing graph theory algorithms. Dynamic behavior adjustment: According to the determined association relationships, the dynamic behavior of the target scene is adjusted. This includes triggering events (such as clicks, hovers), showing / hiding elements, changing element attributes, etc. It is convenient to accurately determine the association relationships between target elements.

[0111] According to some embodiments of the present invention, the second identification module includes:

[0112] The second building module is used to build an audio recognition model;

[0113] The fifth determination module is used to perform audio recognition on the target content, obtain a number of audio clips, determine the frequency spectrum characteristics of each audio clip; input the frequency spectrum characteristics into the audio recognition model, and output audio information.

[0114] The working principle and beneficial effects of the above technical solution are as follows: audio recognition is performed on the target content to obtain several audio clips, and the spectrum characteristics of each audio clip are determined; the extracted spectrum characteristics are input into a pre-trained audio recognition model for inference. The model will output the audio information corresponding to each audio clip.

[0115] According to some embodiments of the present invention, the second building block includes:

[0116] The labeling module is used to obtain the sample spectrum features corresponding to the sample audio clips, build an audio feature training set, and label each audio clip with an audio label;

[0117] The training module is used to build an audio recognition model. The audio feature training set is input into the audio recognition model for training. Based on the difference between the output result and the annotated audio label, the model parameters of the audio recognition model are continuously adjusted until the output result is consistent with the annotated audio label, indicating that the training is completed and a trained audio recognition model is obtained.

[0118] Working principle and beneficial effects of the above technical solution: Collect audio clips from various sources (such as public datasets, self-recording, etc.) as samples. Spectral feature extraction: Use signal processing tools or libraries (such as Python's librosa library) to preprocess and extract spectral features of audio clips. Spectral features include MFCC, spectrogram, CQT (constant Q transform) coefficients, etc. Manually or automatically label the audio clips with audio tags according to their content. For speech recognition tasks, use speech recognition software or manual transcription to generate text labels; for audio event classification tasks, it is necessary to determine the event category to which the audio belongs based on the audio content. Construct a training set: Combine the extracted spectral features and the labeled audio tags to construct an audio feature training set. The training set will be divided into a training subset and a validation subset for model evaluation and tuning during the training process. Model construction: Select a suitable machine learning or deep learning model to build an audio recognition model. Such as a deep learning model (such as CNN, RNN, LSTM, Transformer, etc.). Model training: Input the audio feature training set into the model to start the training process. During the training process, the model learns how to map the input spectral features to the corresponding audio labels. Loss calculation and back propagation: In each iteration, the difference (i.e., loss) between the model's output and the labeled audio labels is calculated, and the loss gradient is propagated back to each layer of the model using the back propagation algorithm to update the model parameters. Model evaluation and tuning: During the training process, the model is regularly evaluated using a validation subset to check whether the model's performance has improved. Tune the model based on the evaluation results, such as adjusting the learning rate, adding regularization terms, changing the model structure, etc. Training stop: When the model's performance reaches the preset stopping condition (such as the loss value no longer decreases, the accuracy on the validation set no longer increases, etc.), stop the training process. Model saving: Save the trained audio recognition model to a file or database for subsequent use.

[0119] According to some embodiments of the present invention, the adjustment module is further used to obtain motion information of a human body in the private room, and perform corresponding adjustment actions according to the motion information of the human body;

[0120] The adjustment module comprises:

[0121] The segmentation module is used to obtain the actual image and the standard image of the private room, and perform image segmentation respectively to obtain a plurality of sub-actual images and a plurality of sub-standard images;

[0122] Matching the sub-actual image with the corresponding sub-standard image, determining the unmatched sub-actual image, and extracting the contour to obtain contour features; matching the contour features with preset human body contour features, and determining the human body image according to the matching results;

[0123] a sixth determination module, configured to obtain key points of a human body according to the human body image, calculate limb vector features of each joint according to the key points of the human body, and determine motion information of the human body according to the limb vector features;

[0124] The execution module is used to execute corresponding adjustment actions according to the action information of the human body.

[0125] The working principle and beneficial effects of the above technical solution are as follows: the actual image is an image of a private room including the user, and the standard image is an image of a private room excluding the user. Use an image matching algorithm (such as feature point matching, template matching, etc.) to compare the sub-actual image with the sub-standard image to find the difference area. Apply a contour detection algorithm (such as Canny edge detection, Hough transform, etc.) to the difference area to extract contour features. Match the extracted contour features with the preset human contour feature library, and determine whether it is a human contour by features such as shape, size, and proportion. Use a deep learning model (such as OpenPose, DeepPose, etc.) to detect key points on the human image and obtain the position of the joint points. Calculate the limb vectors based on the position of the joint points, which describe the relative position and relationship between the various parts of the human body. Based on the limb vector features, combined with the action recognition algorithm or model, determine the action information of the human body. According to the action information of the human body, perform corresponding adjustment actions. The adjustment actions include lighting adjustment, sending reminder information to the waiter, etc. It is convenient to make corresponding adjustments according to the user's behavior and improve the safety of the user in the private room.

[0126] According to some embodiments of the present invention, the acquisition module includes:

[0127] The first acquisition module is used to:

[0128] Based on the temperature and humidity sensors installed in multiple locations in the private room, the temperature and humidity information is obtained;

[0129] Acquire odor information based on odor sensors installed in multiple locations in the private room;

[0130] Determine the environmental parameters of the current environment based on temperature, humidity and odor information;

[0131] The second acquisition module is used to acquire audio information of the current environment as somatosensory information based on audio sensors set at multiple locations in the private room.

[0132] The working principle and beneficial effects of the above technical solution are as follows: Based on the temperature and humidity sensors installed at multiple locations in the private room, the temperature and humidity information is obtained; Based on the odor sensors installed at multiple locations in the private room, the odor information is obtained; Based on the temperature and humidity information and the odor information, the environmental parameters of the current environment are determined; The second acquisition module is used to obtain the audio information of the current environment as the somatosensory information based on the audio sensors installed at multiple locations in the private room. The detection of the temperature and humidity information, odor information and audio information of the current environment is realized.

[0133] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A somatosensory interactive immersive experience room system, characterized in that: include: The first determination module is used to receive the demand keywords input by the user and display the recommended content according to the demand keywords; A playback module, used to receive and play target content selected by a user from the recommended content; The recognition module is used to recognize the scene of the target content and determine the target environment parameters and target body sensory information; The acquisition module is used to obtain the environmental parameters and somatosensory information of the current environment; An adjustment module, used to perform environmental adjustment according to target environmental parameters and environmental parameters, and perform somatosensory adjustment according to target somatosensory information and somatosensory information; The playback module comprises: Glasses-free 3D surround screen; A determination module, used to determine whether the number of target contents selected by the user from the recommended contents is 2, and when it is determined that the number is 2, generate instruction information for dividing the page of the naked-eye 3D surround screen; The second determination module is used for: Determine the first target content and the second target content included in the selected target content; obtain first element information of the first target content and second element information of the second target content; calculate the element difference value between the first target content and the second target content according to the first element information and the second element information; The element difference value is compared with the preset segmentation threshold. When it is determined that the element difference value is less than the preset segmentation threshold, the naked-eye 3D surround screen is equally divided; otherwise, the preset difference value-division strategy data table is queried according to the element difference value to determine the corresponding division strategy, and the naked-eye 3D surround screen is divided according to the division strategy; Play the target content according to the division result of the naked-eye 3D surround screen; The identification module comprises: The second identification module is used for: Performing audio recognition on the target content to obtain audio information; Querying a preset audio-body sensing data table according to the audio information to determine the target body sensing information; The second identification module includes: The second building module is used to build an audio recognition model; The fifth determination module is used to perform audio recognition on the target content, obtain a number of audio clips, determine the frequency spectrum characteristics of each audio clip; input the frequency spectrum characteristics into the audio recognition model, and output audio information; The second building block comprises: The labeling module is used to obtain the sample spectrum features corresponding to the sample audio clips, build an audio feature training set, and label each audio clip with an audio label; The training module is used to build an audio recognition model. The audio feature training set is input into the audio recognition model for training. Based on the difference between the output result and the annotated audio label, the model parameters of the audio recognition model are continuously adjusted until the output result is consistent with the annotated audio label, indicating that the training is completed and a trained audio recognition model is obtained.

2. The somatosensory interactive immersive experience room system according to claim 1, characterized in that: The first determining module includes: The first building module is used to build a recommendation system; the recommendation system includes a title, description, preset keywords and recommendation level; The matching module is used to receive the demand keywords input by the user, match the demand keywords with preset keywords, and determine the corresponding content as the recommended content according to the matching results.

3. The somatosensory interactive immersive experience room system according to claim 1, characterized in that: The identification module further includes: The first identification module is used for: Perform image recognition on the target content and determine the target scene; Determine the corresponding target temperature and humidity information and target odor according to the target scene; determine the target environmental parameters according to the target temperature and humidity information and target odor.

4. The somatosensory interactive immersive experience room system according to claim 3, characterized in that: The first identification module includes: The third determination module is used to: Annotate element boxes in the image of the target content based on a preset trained element recognition model; Perform convolution processing on each element frame to determine the overall convolution feature of the element frame; perform pooling processing on the overall convolution feature, perform multiple levels of residual processing on the overall convolution feature after pooling, and then perform pooling processing again to obtain the overall feature of the element frame; segment the element frame according to the overall feature, determine a number of local features, use the several local features as the feature information corresponding to each element frame, input the feature information corresponding to each element frame into the fully connected network in the element recognition model for channel connection processing, and obtain a feature map; The fourth determination module is used for: Determine the target element corresponding to each element box according to the feature map; Determine the association relationship between target elements; the association relationship includes position relationship and combination relationship; Determine the weight coefficient of each element according to the association relationship between the target elements; Determine the target scene based on the target element and the corresponding weight coefficient.

5. The somatosensory interactive immersive experience room system according to claim 4, characterized in that: The fourth determination module determines the association relationship between target elements, including: A generation module is used to obtain page element data, scene business marketing data and scene space data; split the page element data, scene business marketing data and scene space data to obtain the corresponding component structure, and generate the corresponding JSON data, drag it to the interface editing area, and generate the target scene general map; The activation module is used to match the target element with the elements in the target scene overall map, activate the pixel points of the corresponding elements in the target scene overall map according to the matching results, and determine the association relationship between the target elements based on the overall association relationship of the target scene overall map and the pixel points of the corresponding elements.

6. The somatosensory interactive immersive experience room system according to claim 1, characterized in that: The adjustment module is also used to obtain the motion information of the human body in the private room and perform corresponding adjustment actions according to the motion information of the human body; The adjustment module comprises: The segmentation module is used to obtain the actual image and the standard image of the private room, and perform image segmentation respectively to obtain a plurality of sub-actual images and a plurality of sub-standard images; Matching the sub-actual image with the corresponding sub-standard image, determining the unmatched sub-actual image, and extracting the contour to obtain contour features; matching the contour features with preset human body contour features, and determining the human body image according to the matching results; a sixth determination module, configured to obtain key points of a human body according to the human body image, calculate limb vector features of each joint according to the key points of the human body, and determine motion information of the human body according to the limb vector features; The execution module is used to execute corresponding adjustment actions according to the action information of the human body.

7. The somatosensory interactive immersive experience room system according to claim 1, characterized in that: The acquisition module comprises: The first acquisition module is used to: Based on the temperature and humidity sensors installed in multiple locations in the private room, the temperature and humidity information is obtained; Acquire odor information based on odor sensors installed in multiple locations in the private room; Determine the environmental parameters of the current environment based on temperature, humidity and odor information; The second acquisition module is used to acquire audio information of the current environment as somatosensory information based on audio sensors set at multiple locations in the private room.

Images