Air conditioner control device, method and storage medium based on facial expression recognition

Abstract

The application provides an air conditioner control device and method based on facial expression recognition and a storage medium, which comprises a shooting module for collecting personnel facial images in a preset range; a processing module for facial expression recognition of the personnel facial images to obtain personnel facial expression information; a temperature and humidity collection module for collecting temperature and humidity values in the preset range; the processing module is further used for generating air conditioner control instructions according to the personnel facial expression information and the temperature and humidity values; and a control module for controlling the air conditioner according to the air conditioner control instructions. The application can recognize and analyze personnel facial expressions through collected personnel facial images, generate air conditioner control instructions through personnel facial expressions, temperature and humidity values, use thermal sensation obtained from facial expression analysis as the basis for temperature regulation, and achieve the purpose of reducing energy consumption under the premise of ensuring indoor personnel comfort requirements, so that the comfortable area is larger.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the field of air conditioner automatic control technology, and particularly relates to an air conditioner control device and method based on facial expression recognition and a storage medium. BACKGROUND

[0002] In order to create a comfortable indoor teaching and learning environment in the campus, as a key equipment for creating indoor thermal and humid environment, the improvement of the intelligent level of the air conditioner has a profound influence on the creation of the indoor thermal and humid environment. Therefore, the present project combines the facial expression proportion with the indoor temperature and humidity parameters to obtain the feedback information of the personnel on the indoor thermal environment, and then controls the opening and closing of the air conditioner, the mode selection and the temperature adjustment by using the infrared remote control principle, so as to finally meet the requirements of the indoor personnel on the thermal comfort and reduce the energy consumption of the air conditioner.

[0003] Education is the key to a century's great undertaking. In recent years, with the continuous development of higher education in China, the state's investment in education has been increasing. There are nearly 2000 universities in China, with more than 23 million students, accounting for 4.4% of the total population. Due to its specific function, the campus building is often characterized by a large number of personnel, a strong cultural atmosphere and a large building energy consumption. The classroom and conference room are the main places for the school to carry out various teaching activities and meetings, and the indoor environment will directly affect the students' efficiency. Studies have shown that a good indoor environment can reduce brain fatigue and improve brain efficiency by 20-35%. As a device for adjusting indoor temperature and humidity, the air conditioner can significantly improve the indoor environment of the classroom and meet the comfort of the people in the room. At the same time, the energy consumption of the air conditioning system accounts for a large proportion of the total building energy consumption. The existing methods for adjusting air conditioning parameters and reducing air conditioning energy consumption include intelligent identification of indoor personnel and changing the design temperature of the air conditioner, but they cannot be timely associated with the comfort of the indoor personnel, and the intelligent level is low. SUMMARY

[0004] The present application mainly relates to the field of air conditioner automatic control technology, and particularly relates to an air conditioner control device and method based on facial expression recognition and a storage medium.

[0005] The technical scheme for solving the above technical problems is as follows: an air conditioner control device based on facial expression recognition, comprising:

[0006] A shooting module is configured to collect personnel facial images within a preset range.

[0007] A processing module is configured to perform facial expression recognition on the personnel facial images to obtain personnel facial expression information.

[0008] A temperature and humidity acquisition module is configured to collect temperature and humidity values within a preset range.

[0009] The processing module is further configured to generate an air conditioner control instruction according to the facial expression information of the person and the temperature value and the humidity value.

[0010] The control module is configured to control the air conditioner according to the air conditioner control instruction.

[0011] The present application has the beneficial effect that the facial expression of the person can be recognized and analyzed through the collected facial image of the person, the air conditioner control instruction can be generated through the facial expression of the person, the temperature value and the humidity value, the thermal sensation obtained through the facial expression analysis is used as the basis for temperature regulation, the comfortable area is larger, and the purpose of reducing energy consumption under the premise of ensuring the comfort requirement of the indoor person is achieved.

[0012] On the basis of the above technical solution, the present application can be further improved as follows.

[0013] Further, in the processing module, the facial expression of the person is recognized from the facial image of the person to obtain facial expression information, specifically:

[0014] The facial expression of the person is recognized from the facial image of the person through a pre-trained feature extractor and a Dlib face recognition open source library to obtain a plurality of key points of face detection;

[0015] The facial image of the person is preprocessed, and the path of the preprocessed facial image of the person is loaded, and the number of faces in the loaded facial image of the person is detected to determine a region of interest according to the number of faces;

[0016] The feature point positions of the facial image of the person are calibrated in the region of interest through a circle method in OpenCV, the feature point positions include eyebrow contours and mouth contours, and the facial expression information of the person is obtained according to the eyebrow contours and the mouth contours, the facial expression information of the person includes angry expression, natural expression, surprised expression and happy expression.

[0017] The beneficial effect of the above further technical solution is that the thermal sensation of the indoor person is obtained through the recognition and analysis of the facial expression, and the indoor thermal environment is adjusted in real time based on this.

[0018] Further, in the processing module, the air conditioner control instruction is generated according to the facial expression information of the person and the temperature value and the humidity value, specifically:

[0019] The number of natural expressions and happy expressions in the facial expression information of the person is counted, the ratio of the number to the total number of persons is calculated, if the ratio is greater than or equal to a preset ratio, a keep current state instruction of the air conditioner or a turn off the air conditioner instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module;

[0020] If the ratio is less than the preset ratio and the current season is summer, it is determined whether the current temperature value and the current humidity value satisfy first preset temperature conditions and first preset humidity conditions, and if both satisfy, a keep current state of air conditioner instruction or a turn off air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0021] If the current temperature value satisfies the first preset temperature condition, and if the current humidity value is greater than the preset humidity value, a turn on air conditioner dehumidification instruction is generated, if the current humidity value is less than or equal to the preset humidity value, a turn off air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0022] If the current temperature value does not satisfy the first preset temperature condition and has not reached the first temperature threshold, a turn off air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0023] If the current temperature value does not satisfy the first preset temperature condition and has reached the first temperature threshold and the current humidity value satisfies the first preset humidity condition, a turn on air conditioner cooling instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0024] If the current temperature value does not satisfy the first preset temperature condition and has reached the first temperature threshold and the current humidity value exceeds the first humidity threshold, a turn on air conditioner cooling and dehumidification instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0025] If the current temperature value does not satisfy the first preset temperature condition and has reached the first temperature threshold and the current humidity value does not exceed the first humidity threshold, a turn on air conditioner cooling instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0026] If the ratio is greater than or equal to the preset ratio, a keep current state of air conditioner instruction or a turn off air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0027] Further, in the processing module, the air conditioner control instruction is generated according to the facial expression information of the person and the temperature value and the humidity value, and is further specifically:

[0028] If the ratio is less than the preset ratio and the current season is not summer, it is determined whether the current temperature value and the current humidity value satisfy second preset temperature conditions and second preset humidity conditions, and if both satisfy, a keep current state of air conditioner instruction or a turn off air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0029] If the current temperature value meets the second preset temperature condition, an air conditioner humidification opening instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0030] If the current temperature value does not meet the second preset temperature condition and is not lower than the second temperature threshold, an air conditioner closing instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0031] If the current temperature value does not meet the second preset temperature condition and is lower than the second temperature threshold, and the current humidity value meets the second preset humidity condition, an air conditioner temperature rising opening instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0032] If the current temperature value does not meet the second preset temperature condition and is lower than the second temperature threshold, and the current humidity value does not meet the second preset humidity condition, an air conditioner temperature rising and humidification opening instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0033] The beneficial effects of the above further technical solutions are that in summer refrigeration and winter heating, a higher or lower target temperature can be set to reduce energy consumption under the premise of ensuring the comfort requirement of indoor personnel, and suitable indoor environment solutions can be provided for different heat adaptability of people.

[0034] Further, the control module is specifically configured to convert the air conditioner control instruction into an infrared signal, and emit the infrared signal to the air conditioner to control the air conditioner.

[0035] Further, the application further comprises an APP module, which is configured to load an APP interface in the smart terminal, so that the smart terminal sends a temperature value and humidity value acquisition request through the APP interface.

[0036] The processing module is configured to send the current temperature value and the current humidity value to the APP interface according to the temperature value and humidity value acquisition request.

[0037] The beneficial effects of the above further technical solutions are that information interaction can be performed through the APP interface.

[0038] Another technical solution for solving the above technical problem is as follows: an air conditioner control method based on facial expression recognition, comprising the following steps:

[0039] Acquiring facial images of personnel in a preset range;

[0040] Performing facial expression recognition on the facial images of the personnel to obtain facial expression information of the personnel;

[0041] acquire temperature values and humidity values within a preset range;

[0042] generate an air conditioner control instruction according to the personnel facial expression information and the temperature values and humidity values;

[0043] control the air conditioner according to the air conditioner control instruction.

[0044] Another technical solution of the present application to solve the above technical problems is as follows: an air conditioner control device based on facial expression recognition, characterized by comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the air conditioner control method based on facial expression recognition is realized.

[0045] Another technical solution of the present application to solve the above technical problems is as follows: a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the air conditioner control method based on facial expression recognition is realized. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 a functional module block diagram of the air conditioner control device provided by the embodiment of the present application;

[0047] Figure 2 a schematic diagram of physical connection provided by the embodiment of the present application;

[0048] Figure 3 a logic diagram of generating an air conditioner control instruction provided by the embodiment of the present application;

[0049] Figure 4 a flowchart of the air conditioner control method provided by the embodiment of the present application;

[0050] Figure 5 a schematic diagram of the APP interface provided by the embodiment of the present application. DETAILED DESCRIPTION

[0051] The principles and characteristics of the present application are described below in conjunction with the drawings, and the examples are only used to explain the present application, and are not used to limit the scope of the present application.

[0052] As shown in Figures 1-2 an air conditioner control device based on facial expression recognition, comprising:

[0053] a shooting module for acquiring personnel facial images within a preset range;

[0054] a processing module for facial expression recognition of the personnel facial images to obtain personnel facial expression information;

[0055] The temperature and humidity collection module is configured to collect temperature and humidity values within a preset range.

[0056] The processing module is further configured to generate an air conditioner control instruction according to the facial expression information of the person and the temperature and humidity values.

[0057] The control module is configured to control the air conditioner according to the air conditioner control instruction.

[0058] Specifically, the processing module can be regarded as a server, and an esp8266 chip is built in the server.

[0059] The temperature and humidity collection module is specifically a DHT11, which is a temperature and humidity sensor with calibrated digital signal output, and has a humidity accuracy of ±5% RH, a temperature accuracy of ±2℃, a humidity range of 5-95% RH, and a temperature range of -20-+60℃.

[0060] The shooting module is a wireless monitoring camera, and the wireless monitoring camera has a 360° pan-tilt rotation without dead angle, can be remotely and real-timely viewed and played back on a computer or a mobile phone, and supports cloud storage service. The wireless monitoring camera can be set to continuously record videos all day long and record videos at regular time intervals, so that the camera records videos during the time when teachers and students have classes or meetings, and does not save videos during other time. The specific parameters of the camera are shown in Table 1.

[0061] Table 1

[0062]

[0063]

[0064] In the above embodiment, the facial expression of the person can be recognized and analyzed through the collected facial image of the person, the air conditioner control instruction can be generated through the facial expression of the person, the temperature value and the humidity value, the thermal sensation obtained through the facial expression analysis is used as the basis for temperature adjustment, the comfortable area is larger, and the purpose of reducing energy consumption under the premise of guaranteeing the comfort requirement of indoor persons is achieved.

[0065] Specifically, the facial expression of the person is recognized through the facial image of the person to obtain the facial expression information of the person, and the facial expression information of the person is specifically:

[0066] The facial expression of the personnel facial image is recognized by the pre-trained feature extractor and the Dlib face recognition open source library, and a plurality of key points of face detection are obtained. Specifically, the pre-trained feature extractor on the network is used to recognize the face by using the Dlib library; shape_predictor_68_face_landmarks.dat is mainly used to obtain the feature extractor, and the feature extractor is get_frontal_face_detector. For face detection, Dlib mainly uses 68 key points of face detection.

[0067] The personnel facial image is preprocessed, and the path of the preprocessed personnel facial image is loaded, and the number of faces in the loaded personnel facial image is detected, and the region of interest is determined according to the number of faces. Specifically, the image is preprocessed, the picture path is loaded, the feature extractor is instantiated, the number of faces in the picture is detected, and the region of interest is obtained, and the face is set. The steps of preprocessing include frame processing, grayscale processing, etc.

[0068] The feature point position of the personnel face in the region of interest is calibrated by the circle method in OpenCV, the feature point position includes eyebrow contour and mouth contour, and the personnel facial expression information is obtained according to the eyebrow contour and the mouth contour, the personnel facial expression information includes angry expression, natural expression, surprised expression and happy expression. Specifically, the detected face is combined with the predictor to mark the 68 key point positions of the face, the circle method in OpenCV is used to mark the feature point positions when calibrating, and the watermark is added at the feature point positions, the content is the need and position of the feature point; according to the detection and result, a judgment index in a region is set, first, the opening ratio of the mouth is calculated, because the distance between the person and the camera is far or near, the size of the face recognition frame is different, so the ratio is selected as the judgment index, before selecting the standard value of the index, a plurality of happy face photos are analyzed, and the average value of the mouth opening ratio when happy is calculated; the data processing method of the person's eyebrows is intercepted, the 5 feature points above the left eyebrow are linearly fitted, and a first function straight line is fitted, and the slope of the fitted straight line is used to represent the inclination degree of the eyebrow; the expression of the face is judged according to the shape of the eyebrow and the mouth, and four expressions of angry, natural, amazing and happy can be obtained.

[0069] OpenCV is a cross-platform computer vision and machine learning software library based on Apache2.0 license (open source) and can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient - composed of a series of C functions and a small amount of C++ classes, while providing interfaces for languages such as Python, Ruby, MATLAB, etc., and implements many general algorithms in image processing and computer vision.

[0070] Dlib is a popular open source library for face recognition.

[0071] The expression recognition criteria are:

[0072] 1. The greater the proportion of the mouth opening distance to the face recognition frame width, the more excited the emotion, which may be very happy or extremely angry.

[0073] 2. The smaller the ratio of the distance between the 17th to 21st or 22nd to 26th feature points to the top of the face recognition frame and the height of the recognition frame, the more the eyebrows are raised, which can represent surprise or happiness. From the angle of the tilt of the eyebrows, the eyebrows are generally raised when happy, and the eyebrows are severely pressed down when angry.

[0074] 3. The degree of eye opening, when people laugh heartily, they will unconsciously squint their eyes, and when they are angry or surprised, they will widen their eyes.

[0075] In the above embodiment, the thermal sensation of the indoor personnel is obtained by recognizing and analyzing the facial expressions, and the indoor thermal environment is adjusted in real time based on this.

[0076] Specifically, as shown in the figure, Figure 3 In the processing module, the air conditioner control instruction is generated according to the personnel facial expression information and the temperature value and the humidity value, specifically:

[0077] The number of natural expressions and happy expressions in the personnel facial expression information is counted, and the ratio of the number to the total number of personnel is calculated. If the ratio is greater than or equal to a preset ratio, a keep air conditioner current state instruction or a turn off air conditioner instruction is generated, and the temperature value and humidity value are continuously collected by calling the temperature and humidity collection module.

[0078] If the ratio is less than the preset ratio, and the current season is summer, it is determined whether the current temperature value and the current humidity value satisfy the first preset temperature condition and the first preset humidity condition. If both conditions are satisfied, a keep air conditioner current state instruction or a turn off air conditioner instruction is generated, and the temperature value and humidity value are continuously collected by calling the temperature and humidity collection module.

[0079] If the current temperature value meets the first preset temperature condition, and if the current humidity value is greater than the preset humidity value, an air conditioner dehumidification instruction is generated, if the current humidity value is less than or equal to the preset humidity value, a close air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0080] If the current temperature value does not meet the first preset temperature condition and the first temperature threshold has not been reached, a close air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0081] If the current temperature value does not meet the first preset temperature condition and the first temperature threshold has been reached and the current humidity value meets the first preset humidity condition, an air conditioner cooling instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0082] If the current temperature value does not meet the first preset temperature condition and the first temperature threshold has been reached and the current humidity value exceeds the first humidity threshold, an air conditioner cooling and dehumidification instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0083] If the current temperature value does not meet the first preset temperature condition and the first temperature threshold has been reached and the current humidity value does not exceed the first humidity threshold, an air conditioner cooling instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0084] If the ratio is greater than or equal to the preset ratio, a keep air conditioner current state instruction or a close air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0085] Specifically, in the processing module, the air conditioner control instruction is generated according to the personnel facial expression information and the temperature value and the humidity value, and specifically:

[0086] If the ratio is less than the preset ratio and the current season is not summer, it is judged whether the current temperature value and the current humidity value meet the second preset temperature condition and the second preset humidity condition, if both meet, a keep air conditioner current state instruction or a close air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0087] If the current temperature value meets the second preset temperature condition, an air conditioner humidification instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0088] If the current temperature value does not meet the second preset temperature condition and is not lower than the second temperature threshold, a close air conditioner instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value;

[0089] If the current temperature value does not satisfy the second preset temperature condition and is lower than the second temperature threshold and the current humidity value satisfies the second preset humidity condition, an open air conditioner temperature rising instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0090] If the current temperature value does not satisfy the second preset temperature condition and is lower than the second temperature threshold and the current humidity value does not satisfy the second preset humidity condition, an open air conditioner temperature rising and humidifying instruction is generated, and the temperature and humidity acquisition module is called to continue to acquire the temperature value and the humidity value.

[0091] In the above embodiment, in summer refrigeration and winter heating, a higher or lower target temperature can be set to achieve the purpose of reducing energy consumption under the premise of ensuring the comfort requirement of indoor personnel, and suitable indoor environment solutions can be provided for different heat adaptability of people.

[0092] Specifically, the control module is specifically configured to convert the air conditioner control instruction into an infrared signal, and transmit the infrared signal to the air conditioner to control the air conditioner.

[0093] The control module includes an infrared emission unit, which respectively encodes infrared signals for each function of the air conditioner terminal, so as to realize control of different functions of the air conditioner.

[0094] The infrared emission unit controls the infrared emission device to emit an infrared signal according to the control instruction of the processing module (server) and the corresponding infrared encoding mode.

[0095] For example, when receiving the instruction "mode1", the infrared signal of the infrared emission unit is encoded to "turn on the air conditioner", and the infrared signal will be emitted after the encoding is completed.

[0096] Specifically, the application further comprises an APP module, which is configured to load an APP interface in the smart terminal, so that the smart terminal sends a temperature value and humidity value acquisition request through the APP interface.

[0097] The processing module is configured to send the current temperature value and the current humidity value to the APP interface according to the temperature value and humidity value acquisition request.

[0098] The APP module is loaded in a PC terminal, a mobile phone terminal and an iPad device terminal, and a user can view current indoor temperature and humidity data through the APP interface, and can also feed back requirements on the interface to adjust the temperature and humidity values.

[0099] Specifically, the processing module (server) establishes communication with the intelligent terminal through the MQTT communication protocol, which is a low-overhead, low-bandwidth instant messaging protocol that can provide real-time and reliable message services for connecting remote devices with minimal code and bandwidth. Therefore, in the transmission of control instructions and temperature and humidity data, the MQTT transmission protocol is used. The esp8266 and the server interact with each other through the MQTT protocol. After a certain period of time, the esp8266 sends a heartbeat packet to the server to indicate a connection state. When the set interval time is reached, the esp8266 collects temperature and humidity data and transmits it to the server, which transmits the temperature and humidity data to the intelligent terminal.

[0100] In the above embodiment, information interaction can be performed through the APP interface.

[0101] Specifically, regarding the esp8266, it is a complete and self-contained WIFI network solution that can operate independently or as a slave on other hosts.

[0102] The esp8266 supports softAP mode, station mode, and softAP+station coexistence mode. Using the esp8266, flexible network topology and networking can be achieved. When the esp8266 is in softAP mode, it acts as a central node of a wireless network, such as a wireless router. When the esp8266 is in station mode, it acts as a terminal node of a wireless network, such as a mobile phone. When the esp8266 is in softAP+station coexistence mode, it can simultaneously act as a central node and a terminal node of a wireless network, such as a mobile phone that provides a hotspot for other devices to connect to. In this design, the mode of the esp8266 is station mode, acting as a terminal node of a wireless network, interacting with the server, temperature and humidity sensor, and infrared emitter.

[0103] When the esp8266 is in station mode, it acts as a terminal node of a wireless network. It can interact with the server and other sensors. When the esp8266 connects to a wireless network central node, it will automatically connect to the server, enabling data interaction with the server.

[0104] As shown in Figure 5 the APP interface is introduced as follows:

[0105] The user enters the App interface, registers a login account first, agrees to each authorization, and can see the current outdoor air temperature and humidity parameters after entering the home page. Meanwhile, the system automatically locates the current position, and manual positioning can be selected when the positioning is inaccurate. After opening the WIFI, the user can see the air conditioning devices that can be controlled in the vicinity of the location, and can select a certain air conditioning device to adjust, such as the 214 office. After connecting with the air conditioning device in the 214 office, the user can enter the room and view the indoor temperature and humidity and the expression proportion of the indoor personnel. The indoor personnel can adjust the temperature to meet the comfort requirement. Common operation problems can be found in the use help, and the online customer service can be contacted when the use help cannot be solved.

[0106] Embodiment 2

[0107] As shown in Figure 4 A face expression recognition-based air conditioner control method, comprising the following steps:

[0108] S1: collecting face images of personnel in a preset range;

[0109] S2: performing face expression recognition on the face images of the personnel to obtain face expression information of the personnel;

[0110] S3: collecting temperature and humidity values in the preset range;

[0111] S4: generating an air conditioner control instruction according to the face expression information of the personnel and the temperature and humidity values;

[0112] S5: controlling the air conditioner according to the air conditioner control instruction.

[0113] Specifically, the face expression recognition on the face images of the personnel to obtain face expression information of the personnel is specifically:

[0114] The face expression recognition on the face images of the personnel is performed by using a pre-trained feature extractor and a Dlib face recognition open source library to obtain a plurality of key points of face detection;

[0115] The face images of the personnel are preprocessed, the path of the preprocessed face images of the personnel is loaded, the number of faces in the loaded face images of the personnel is detected, and a region of interest is determined according to the number of faces;

[0116] The feature point positions of the face of the personnel are demarcated in the region of interest by using a circle method in OpenCV, the feature point positions include eyebrow contours and mouth contours, face expression information of the personnel is obtained according to the eyebrow contours and the mouth contours, and the face expression information of the personnel includes angry expression, natural expression, surprised expression and happy expression.

[0117] Embodiment 3:

[0118] The air conditioner control device based on facial expression recognition, characterized in comprising a memory, a processor and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the air conditioner control method is realized.

[0119] Embodiment 4:

[0120] A computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by the processor, the air conditioner control method is realized.

[0121] It should be noted that, in this paper, such as the first and second relationship terms are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or equipment.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above described device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0123] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the above described device embodiment is only schematic, for example, the division of unit is only a logical function division, and actual implementation can have another division way, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0124] The unit described as a separate component can be or can not be physically separated, and the component displayed as a unit can be or can not be a physical unit, that is, it can be located in one place, or it can be distributed to multiple network units. According to the actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0125] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0126] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0127] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An air conditioner control device based on facial expression recognition, characterized by, The method comprises the following steps: a photographing module is used to collect facial images of people within a preset range; a processing module is used to perform facial expression recognition on the facial images of people to obtain facial expression information of people; a temperature and humidity collection module is used to collect temperature values and humidity values within a preset range; the processing module is further used to generate air conditioner control instructions according to the facial expression information of people and the temperature values and humidity values; a control module is used to control an air conditioner according to the air conditioner control instructions; in the processing module, the facial expression recognition on the facial images of people to obtain the facial expression information of people is specifically performed as follows: a facial expression recognition is performed on the facial images of people by using a pre-trained feature extractor and a Dlib open source library for human face recognition to obtain a plurality of key points of human face detection; the facial images of people are preprocessed, a path of the preprocessed facial images of people is loaded, and the number of human faces in the loaded facial images of people is detected to determine a region of interest according to the number of human faces; feature point positions of a human face in the region of interest are calibrated by using a circle method in OpenCV, the feature point positions include eyebrow contours and a mouth contour, and facial expression information of a human face is obtained according to the eyebrow contours and the mouth contour, the facial expression information of a human face includes an angry expression, a natural expression, a surprised expression and a happy expression; in the processing module, the air conditioner control instructions are generated according to the facial expression information of people and the temperature values and humidity values, and the specific process is as follows: the number of natural expressions and happy expressions in the facial expression information of people is counted, a ratio of the number to the total number of people is calculated, if the ratio is greater than or equal to a preset ratio, a keep current state instruction of an air conditioner or a turn off air conditioner instruction is generated, and the temperature values and humidity values are continuously collected by calling the temperature and humidity collection module; if the ratio is less than the preset ratio and the current season is summer, it is determined whether the current temperature value and the current humidity value meet first preset temperature conditions and first preset humidity conditions, if both meet, the keep current state instruction of the air conditioner or the turn off air conditioner instruction is generated, and the temperature values and humidity values are continuously collected by calling the temperature and humidity collection module; if the current temperature value meets the first preset temperature conditions, if the current humidity value is greater than a preset humidity value, a turn on air conditioner dehumidification instruction is generated, if the current humidity value is less than or equal to the preset humidity value, a turn off air conditioner instruction is generated, and the temperature values and humidity values are continuously collected by calling the temperature and humidity collection module; if the current temperature value does not meet the first preset temperature conditions and a first temperature threshold has not been reached, a turn off air conditioner instruction is generated, and the temperature values and humidity values are continuously collected by calling the temperature and humidity collection module; if the current temperature value does not meet the first preset temperature conditions and the first temperature threshold has been reached and the current humidity value meets the first preset humidity conditions, a turn on air conditioner cooling instruction is generated, and the temperature values and humidity values are continuously collected by calling the temperature and humidity collection module. If the current temperature value does not satisfy the first preset temperature condition, the first temperature threshold has been reached, and the current humidity value exceeds the first humidity threshold, an open air conditioner cooling and dehumidifying instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the current temperature value does not satisfy the first preset temperature condition, the first temperature threshold has been reached, and the current humidity value does not exceed the first humidity threshold, an open air conditioner cooling instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the ratio is greater than or equal to the preset ratio, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; In the processing module, the air conditioner control instruction is generated according to the personnel facial expression information and the temperature value and the humidity value, and is specifically: If the ratio is less than the preset ratio, and the current season is not summer, it is judged whether the current temperature value and the current humidity value satisfy the second preset temperature condition and the second preset humidity condition, if both satisfy, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the current temperature value satisfies the second preset temperature condition, an open air conditioner humidifying instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the current temperature value does not satisfy the second preset temperature condition and is not lower than the second temperature threshold, a close air conditioner instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the current temperature value does not satisfy the second preset temperature condition and is lower than the second temperature threshold, and the current humidity value satisfies the second preset humidity condition, an open air conditioner heating instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module; If the current temperature value does not satisfy the second preset temperature condition and is lower than the second temperature threshold, and the current humidity value does not satisfy the second preset humidity condition, an open air conditioner heating and humidifying instruction is generated, and the temperature value and the humidity value are continuously collected by calling the temperature and humidity collection module.

2. The air conditioner control device according to claim 1, characterized by The control module is specifically used for converting the air conditioner control instruction into an infrared signal, and emitting the infrared signal to the air conditioner to control the air conditioner.

3. The air conditioner control device according to claim 1, wherein An APP module is further included, and the APP module is used for loading an APP interface in a smart terminal, so that the smart terminal sends a temperature value and humidity value acquisition request through the APP interface; The processing module is used for sending the current temperature value and the current humidity value to the APP interface according to the temperature value and humidity value acquisition request.

4. An air conditioner control method based on facial expression recognition, characterized by, The method comprises the following steps: Collecting personnel facial images in a preset range; Performing facial expression recognition on the personnel facial images to obtain personnel facial expression information; Collecting temperature values and humidity values in a preset range; Generating an air conditioner control instruction according to the personnel facial expression information and the temperature values and humidity values; Controlling the air conditioner according to the air conditioner control instruction; The personnel facial expression information is obtained by performing facial expression recognition on the personnel facial images, and specifically: The facial expression of the personnel is recognized through a pre-trained feature extractor and a Dlib face recognition open source library, and a plurality of key points of face detection are obtained; The personnel face image is preprocessed, and a path of the preprocessed personnel face image is loaded, and the number of faces in the loaded personnel face image is detected, and a region of interest is determined according to the number of faces; Feature point positions of the personnel face are calibrated in the region of interest through a circle method in OpenCV, the feature point positions include eyebrow contours and mouth contours, and personnel facial expression information is obtained according to the eyebrow contours and the mouth contours, the personnel facial expression information includes angry expression, natural expression, surprised expression and happy expression; An air conditioner control instruction is generated according to the personnel facial expression information and temperature and humidity values, specifically: The number of natural expressions and happy expressions in the personnel facial expression information is counted, and a ratio of the number to the total number of personnel is calculated, if the ratio is greater than or equal to a preset ratio, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; If the ratio is less than the preset ratio, and the current season is summer, it is judged whether the current temperature value and the current humidity value meet the first preset temperature condition and the first preset humidity condition, if both meet, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value meets the first preset temperature condition, and if the current humidity value is greater than a preset humidity value, an open air conditioner dehumidification instruction is generated, if the current humidity value is less than or equal to the preset humidity value, a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value does not meet the first preset temperature condition and has not reached a first temperature threshold, a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value does not meet the first preset temperature condition and has reached the first temperature threshold, and the current humidity value meets the first preset humidity condition, an open air conditioner cooling instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value does not meet the first preset temperature condition and has reached the first temperature threshold, and the current humidity value exceeds a first humidity threshold, an open air conditioner cooling and dehumidification instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value does not meet the first preset temperature condition and has reached the first temperature threshold, and the current humidity value does not exceed the first humidity threshold, an open air conditioner cooling instruction is generated, and temperature and humidity values are continuously collected; If the ratio is greater than or equal to the preset ratio, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; An air conditioner control instruction is generated according to the personnel facial expression information and temperature and humidity values, and specifically: If the ratio is less than the preset ratio, and the current season is not summer, it is judged whether the current temperature value and the current humidity value meet the second preset temperature condition and the second preset humidity condition, if both meet, a keep current state instruction of the air conditioner or a close air conditioner instruction is generated, and temperature and humidity values are continuously collected; If the current temperature value meets the second preset temperature condition, an air conditioner humidification opening instruction is generated, and the temperature value and the humidity value are continuously collected; If the current temperature value does not meet the second preset temperature condition and is not lower than the second temperature threshold, an air conditioner closing instruction is generated, and the temperature value and the humidity value are continuously collected; If the current temperature value does not meet the second preset temperature condition, is lower than the second temperature threshold, and the current humidity value meets the second preset humidity condition, an air conditioner temperature rising opening instruction is generated, and the temperature value and the humidity value are continuously collected; If the current temperature value does not meet the second preset temperature condition, is lower than the second temperature threshold, and the current humidity value does not meet the second preset humidity condition, an air conditioner temperature rising and humidification opening instruction is generated, and the temperature value and the humidity value are continuously collected.

5. An air conditioner control device based on facial expression recognition, characterized by, The computer program product comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the computer program product implements the air conditioner control method based on facial expression recognition.

6. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 5. When the computer program is executed by the processor, the computer program implements the air conditioner control method based on facial expression recognition.

Images