Method and system for controlling 4D cinema dynamic seats based on face recognition

Abstract

The invention discloses a control method and system for 4D cinema dynamic seats based on face recognition. The method comprises the steps that S110, face images of audiences are collected in real time; s120, carrying out the classification and recognition of the obtained face image information in real time through a trained face recognition model, and obtaining a recognition result; and step S130, adjusting the seat action in real time according to the recognition result. According to the technical scheme, the face image of the audience of each seat is collected in real time, the AI face recognition model classifies and recognizes the obtained face image information in real time, the age and expression information of the audience is obtained, and the character features are judged according to the age and expression information so as to control the motion of the dynamic seats of the 4D cinema.

Description

Technical Field

[0001] This application relates to the technical field of dynamic seats, and specifically relates to a control method and system for a 4D cinema dynamic seat based on face recognition. Background Art

[0002] With the rapid development of technology, the display methods in museums are gradually changing from traditional static exhibitions to diversified and interactive directions. Among them, as an important interactive experience facility in museums, 4D cinemas are highly favored because they can bring audiences immersive movie-watching experiences. However, while enjoying this immersive experience, how to ensure movie-watching safety, especially for special groups such as children, the elderly, and audiences in special states (such as sudden discomfort), has become an urgent problem to be solved.

[0003] The motion control of traditional 4D cinema dynamic seats is mostly based on preset scripts and unified signal control, providing a unified dynamic effect, lacking the ability to intelligently adjust according to the individual differences of audiences. Excessive intense dynamic effects may bring safety hazards, such as sudden forward tilting, backward tilting, or vibration, which may cause children, the elderly, or audiences in sudden discomfort to be injured when the seats move violently, increasing the movie-watching risk. Therefore, developing a system that can automatically adjust the seat motion mode according to the characteristics of the audience is of great significance for improving the movie-watching safety and comfort of 4D cinemas. Summary of the Invention

[0004] In view of the above situation, embodiments of this application propose a control method and system for a 4D cinema dynamic seat based on face recognition. The technical solution is to collect the face images of the audiences on each seat in real time, and the AI face recognition model classifies and recognizes the obtained face image information in real time to obtain the age and expression information of the audiences, and then judges the characteristics of the people accordingly to control the motion of the 4D cinema dynamic seat.

[0005] In a first aspect, embodiments of this application provide a control method for a 4D cinema dynamic seat based on face recognition, and the method includes:

[0006] Step S110: Collect the facial images of the audiences in real time;

[0007] Step S120: Use the trained face recognition model to classify and recognize the obtained facial image information in real time to obtain the recognition result;

[0008] Step S130: Adjust the seat actions in real time according to the recognition result.

[0009] In a second aspect, an embodiment of the present application further provides a control system for a 4D cinema dynamic seat based on face recognition. The control system includes a seat control terminal, an image collector, and a motor control system provided on each seat in the cinema. The motor control system is connected to the seat control terminal through a controller area network link. The seat control terminal is configured with a face recognition model. The image collector transmits face picture data to the face recognition model, and the face recognition model sends the recognition result to the seat control terminal. The seat control terminal obtains seat position parameters based on the age and current mental state of the audience, and automatically transmits the seat position parameters to the motor control system, so that the motor control system automatically adjusts the seat according to the seat position parameters.

[0010] The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects:

[0011] 1. Improve the safety of watching movies: By intelligently identifying the characteristics of the audience and adjusting the seat movement mode, the risk of children, the elderly, and audiences with physical discomfort being injured due to violent seat movement is effectively avoided.

[0012] 2. Enhance the comfort of watching movies: Provide a personalized movie-watching experience for different audiences, meet the needs of audiences of different ages and states, and improve the overall movie-watching comfort.

[0013] 3. Improve the operation efficiency: Automatic control reduces the need for manual intervention, reduces the operation cost, and improves the management efficiency of the cinema.

[0014] 4. Promote technological innovation: This solution integrates cutting-edge technologies such as AI face recognition, deep learning, and the Internet of Things, providing new ideas for the intelligent upgrade of 4D cinemas in museums. Description of the Drawings

[0015] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0016] Figure 1 A flowchart showing the control method of a 4D cinema dynamic seat based on face recognition according to an embodiment of the present application;

[0017] Figure 2 A schematic diagram showing the monitoring process of the movie-watching process of a 4D cinema dynamic seat based on face recognition according to the present application. Detailed Embodiments

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0019] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0020] The embodiments of the present invention take into account that the motion control of traditional 4D cinema dynamic seats is mostly based on preset programs and unified signal control, and lacks the ability to make intelligent adjustments based on individual differences of the audience. Excessively intense dynamic effects may bring safety hazards, such as sudden forward tilt, backward lean or vibration, which may cause children, the elderly or audiences who suddenly feel unwell to be injured when the seats move violently, increasing the risk of watching movies.

[0021] Based on this, an embodiment of the present application proposes a solution, which collects facial images of the audience in each seat in real time, and the AI ​​face recognition model classifies and identifies the acquired facial image information in real time, obtains the audience's age and expression information, and judges the character characteristics based on this to control the movement of the dynamic seats in the 4D cinema. Technical solution.

[0022] Figure 1 A method for 4D cinema dynamic seats based on face recognition according to an embodiment of the present application is shown. Figure 1 It can be seen that the present application at least includes steps S110 to S140:

[0023] Step S110: collecting the audience's facial images in real time.

[0024] In this application, a camera (image acquisition unit) installed in the theater is used to collect the audience's facial image information in real time; a high-definition camera is deployed in the 4D theater to cover the entire viewing area. The camera captures the audience's facial images in real time, and collects a large number of facial images including the elderly, children, people in physical discomfort and other age groups in real time.

[0025] Step S110 further includes:

[0026] Preprocess the acquired face image.

[0027] The acquired face image is preprocessed, further including steps such as face detection, alignment, and normalization, to eliminate interference factors such as noise and illumination changes in the image and improve the accuracy of the algorithm;

[0028] Step S120: Use the trained AI face recognition model to classify and recognize the acquired face image information in real time to obtain the recognition result.

[0029] Specifically: Analyze the collected images through the face recognition algorithm to identify features such as the age, expression, and texture of the audience. For example, identify the elderly and children through facial features such as skin texture, wrinkle distribution, facial proportion, and contour, or judge the current physical state of the audience by closed eyes, relaxed facial muscles, and body posture.

[0030] The implementation process of step S120 includes:

[0031] First step, data collection and preprocessing: Collect a large amount of face image data including the elderly, children, people in poor physical condition, and people of other age groups, and preprocess the collected images, including steps such as face detection, alignment, and normalization, to eliminate interference factors such as noise and illumination changes in the images and improve the accuracy of the algorithm.

[0032] Second step, facial feature extraction: Use a deep learning model (such as a convolutional neural network CNN) to extract features from the preprocessed face images (including skin texture, wrinkle distribution, facial proportion, facial muscles, and posture, etc.) to generate high-dimensional feature vectors.

[0033] Third step, model training and classification processing: Use the preprocessed data to train the deep learning model so that it can learn the feature representations and states of faces of different age groups, and use the designed classifier to map the extracted feature vectors to different age group categories (the elderly, children) and physical discomfort states, and continuously optimize the algorithm parameters to improve the classification accuracy.

[0034] Fourth step, face recognition and differentiation: After the model training is completed, the algorithm automatically performs face detection, feature extraction, and classification, and finally outputs the recognition result.

[0035] The AI face recognition model adopted in this application is such as a convolutional neural network (CNN) or a recurrent neural network (RNN), etc. By inputting through multiple network structures, it automatically extracts high-level features in the image. Using hyperparameter tuning, such as learning rate, batch size, and regularization strength, will significantly affect the facial performance, and realize the classification and recognition of faces.

[0036] The trained AI face recognition model specifically includes: a training data acquisition step, an AI face recognition model establishment step, an AI face model training step, and an AI face model optimization step.

[0037] The training data acquisition step further includes:

[0038] Build an image library, which includes a large amount of face image data of the elderly, children, people in poor physical condition, and people of other age groups collected. Preprocess the collected images, including steps such as face detection, alignment, and normalization, to eliminate interference factors such as noise and illumination changes in the images, improve the accuracy of the algorithm, and then obtain a training set and a validation set, and label each image in the training set with an initial facial expression category and age label;

[0039] For the faces in the pictures of the training set, and perform key point annotation on them: Face detection and key point annotation can be carried out with the help of the Dlib library. This library provides a series of functions related to fields such as machine learning, numerical calculation, graph model algorithms, and image processing. Since different faces may have different poses, which have a great impact on the accuracy of expression recognition, key point annotation is required to improve the accuracy of expression prediction. The face key points include information such as facial features such as skin texture, wrinkle distribution, facial proportion, and contour.

[0040] The specific steps for building the AI face recognition model in this application are as follows:

[0041] Construct a face expression recognition model based on the convolutional neural network CNN, and input the images in the training set into this model respectively to obtain the predicted emotion category of each image in the training set;

[0042] The face expression recognition model is a depthwise separable convolutional neural network CNN. The front end of this network can be a combination of multiple convolutional layers, pooling layers, and fully connected layers, and its backend is a loss layer such as softmax. The specific structure of the convolutional neural network CNN is as follows:

[0043] Convolutional layer: Its input is the face area located in the picture and the key points extracted from the face area, and new features are obtained through transformation output;

[0044] Pooling layer: The pooling layer can map multiple values to one value. This layer can not only further strengthen the learned new features, but also make the output feature space smaller. The output of this layer can be used as the input of the convolutional layer or the fully connected layer again;

[0045] Convolutional layer: Its input is the new features generated by the previous pooling layer, and new features are obtained through transformation output;

[0046] Dropout layer: This layer can prevent the convolutional neural network from overfitting. Its output can be used as the input of the fully connected layer or the pooling layer. Fully connected layer: It makes a linear transformation on the input of the anti-overfitting layer and projects the learned features into a better subspace to facilitate attribute prediction. The Dropout layer prevents the convolutional neural network from overfitting

[0047] Softmax layer: responsible for calculating the error between the predicted attribute category and the input attribute category.

[0048] Output layer: mainly responsible for outputting the category of the expression in the picture. Generally speaking, the located face region and the key points extracted from the face region are used as inputs and fed into the convolutional layer, pooling layer and fully connected layer, and then some anti-overfitting operations are performed.

[0049] Through the multi-layer design of the above convolutional neural network, the accuracy of expression recognition in the video is guaranteed.

[0050] AI face model training steps, used to train the hyperparameters in the convolutional neural network through the collected and labeled data. By comparing the predicted emotion category of each image with its face expression category label respectively, and minimizing the loss between the two, the network parameters in the model are updated until the model maintains stable emotion prediction performance on the training set, and the model is saved;

[0051] AI face model optimization steps: use the validation set to test the model until a model with excellent performance on the validation set is obtained;

[0052] Input the real-time captured face images of the audience into the trained AI face recognition model, automatically perform face detection, feature extraction and classification using the AI face recognition model, obtain the recognition result, and finally output the recognition result, which includes age (old people, children) and face expressions (happy, painful, etc.). The AI face recognition model of this application can recognize age and face expressions.

[0053] Step S130, according to the recognition result, adjust the seat movement in real time according to the recognition result.

[0054] Specifically: according to the result of face recognition, judge whether the current movement mode of the seat is suitable for the current person. For example, when an old person or a child is recognized, the system may reduce the vibration intensity of the seat or limit the moving range of the seat to adapt to the tolerance of the old person or the child. If it is recognized that the audience's body is in a discomfort state, the system may pause or reduce the special effects of the seat, confirm the audience's body state, and avoid accidents.

[0055] Refer to Figure 2 Throughout the movie-watching process, continuous monitoring is carried out, specifically: real-time monitoring and feedback, and the seat movement strategy is dynamically adjusted according to the audience's reaction and the actual situation. At the same time, the recognition algorithm and control logic are continuously optimized through data analysis to improve the accuracy and stability of the system.

[0056] The present application also provides a control system for a 4D cinema dynamic seat based on face recognition, including: a central control server set in the cinema, and the central control server is connected to the control system of each dynamic seat through a switch. The control system includes a seat control terminal, an image collector, and a motor control system arranged on each seat in the cinema. The motor control system can be connected to the seat control terminal through a Controller Area Network (Can) link. The seat control terminal can transmit the parameters for adjusting the car seat through the Can link to the motor control system 13, and the motor control system 13 can adjust the seat according to the parameters.

[0057] The image collector includes a high-definition camera deployed inside the 4D cinema, covering the entire viewing area. The camera captures the audience's picture in real time and transmits the face picture data to the face recognition module. The AI face recognition model can be configured in the central control server or the seat control terminal.

[0058] The image collector transmits the audience's face picture data to the face recognition module, and the face recognition module sends the recognition results (age, mental state) to the seat control terminal. The seat control terminal obtains the seat position parameters according to the audience's age and current mental state, and automatically transmits the seat position parameters to the motor control system, so that the motor control system automatically adjusts the seat according to the seat position parameters.

[0059] The motor control system includes a motor controller and a motor unit. The seat control terminal of each seat is electrically connected to the motor controller and the motor unit of the seat in sequence. The seat control terminal of each seat is electrically connected to the motor controller of the seat through a control line. The sensor unit of each seat is electrically connected to the seat control terminal of the seat.

[0060] The image collector transmits the audience's face picture data to the face recognition module in real time, and the face recognition module sends the recognition results (age, expression) to the seat control terminal in real time. The control terminal obtains the seat position parameters according to the audience's age and current mental state, and sends the seat position parameters to the motor unit of the seat. The motor controller generates an action instruction according to the seat position parameters and sends the action instruction to the motor unit, and the motor unit drives the seat to make an action.

[0061] The seat is also equipped with a sensor unit for measuring the voltage, current, temperature of each motor of the motor unit and the alarm signal of the motor controller, etc., and sending the measurement results to the seat control terminal.

[0062] In another embodiment, the AI face recognition model is configured in a remote AI face recognition module. The AI face recognition module is electrically connected to the central control server through a network, and sends the recognition result to the central control server, which then sends it to the control systems of each seat.

[0063] The solution of this application realizes intelligent recognition and differential control. For the first time, the AI face recognition technology is applied to the control of 4D cinema dynamic seats, realizing differential seat movement control according to the characteristics of the audience.

[0064] The solution of this application has real-time feedback and dynamic adjustment: a real-time feedback mechanism is established, which can dynamically adjust the seat movement strategy according to the characteristics and status of the audience and the actual situation, improving the adaptability and flexibility of the system.

[0065] The solution of this application adopts the integration of multi-field technologies: this solution integrates multi-field technologies such as AI, deep learning, and the Internet of Things, demonstrating the great potential of technological innovation in improving the service quality and audience experience of museums.

[0066] In summary, the technical solution proposed in this patent for AI face recognition based on camera-captured images and controlling the movement of 4D cinema dynamic seats has significant advantages in improving the safety, comfort, and operation efficiency of the movie viewing, and is an important innovative achievement in the intelligent upgrade of museums.

[0067] The above are only the embodiments of this application and are not intended to limit this application. For those skilled in the art, various changes and modifications can be made to this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims

1. A control method for 4D cinema dynamic seats based on face recognition, characterized in that: Step S110: collecting facial images of audience members in real time; Step S120: using the trained face recognition model to classify and recognize the acquired facial image information in real time to obtain a recognition result; Step S130: adjusting the seat movement in real time according to the recognition result.

2. The control method of the 4D cinema dynamic seat according to claim 1, characterized in that: Step S130 also includes obtaining seat position parameters according to the recognition result, and adjusting the seat in real time according to the seat position.

3. The control method of the 4D cinema dynamic seat according to claim 1, characterized in that: Seat position parameters include vibration intensity or limiting the seat's range of motion.

4. The control method of the 4D cinema dynamic seat according to claim 1, characterized in that: Step S120 also includes: a training data acquisition step, a face recognition model establishment step, a face model training step, and a face recognition model optimization step.

5. The control method of the 4D cinema dynamic seat as claimed in claim 2, characterized in that: The training data acquisition step further comprises: Establish an image library, which includes the collected facial image data of the elderly and children in physical discomfort; The collected images are preprocessed to obtain the training set and validation set, and each image in the training set is given an initial facial expression category and an age label.

6. The control method of the 4D cinema dynamic seat according to claim 2, characterized in that: The steps to build a face recognition model include: Construct a facial expression recognition model based on convolutional neural network CNN; Used to train hyperparameters in convolutional neural networks by collecting and annotating data.

7. A control system for 4D cinema dynamic seats based on face recognition, characterized in that: The control system includes a seat control terminal, an image collector, and a motor control system arranged on each seat in the theater. The motor control system is connected to the seat control terminal through a controller local area network link. The seat control terminal is configured with a face recognition model. The image collector transmits face picture data to the face recognition model, and the face recognition model sends the recognition result to the seat control terminal. The seat control terminal obtains seat position parameters according to the age and current mental state of the audience, and automatically transmits the seat position parameters to the motor control system, so that the motor control system automatically adjusts the seat according to the seat position parameters.

8. The control system according to claim 7, characterized in that: It also includes a central control server arranged in the cinema, and the central control server is connected with the control system of each dynamic seat through a switch.

9. The control system according to claim 8, characterized in that: The motor control system includes a motor controller and a motor unit. The seat control terminal of each seat is electrically connected to the motor controller and the motor unit of the seat in sequence. The seat control terminal of each seat is electrically connected to the motor controller of the seat through a control line. The sensor unit of each seat is electrically connected to the seat control terminal of the seat.

Images