A psychological state recognition system and method based on facial images and physiological information

Abstract

The present invention proposes a psychological state recognition system and method based on facial images and physiological information. The psychological state recognition system based on facial images and physiological information includes a cabin, multiple types of image acquisition devices, a biosensor group and a data processing platform; wherein the multiple types of image acquisition devices and biosensor groups are arranged in the cabin, and the image acquisition devices and biosensor groups are connected to the data processing platform for data communication via wireless communication. The psychological state recognition system and method based on facial images and physiological information proposed by the present invention have the characteristics of timely response when problems occur in data transmission, ensuring the continuity and accuracy of psychological state recognition, etc.

Description

Technical Field

[0001] The invention proposes a psychological state recognition system and method based on facial images and physiological information, belonging to the technical field of psychological state recognition. Background Art

[0002] As an outstanding representative of today's high technology, the psychological state recognition system combines face recognition technology and sophisticated image analysis technology, and is specifically used to accurately detect and judge an individual's psychological state. The workflow of this system starts with a high-precision camera, which can keenly capture the user's facial expressions and those subtle movements that are not easy to detect. The above movements and expressions may only flash for a moment for ordinary people, but for the psychological state recognition system, they are valuable clues to interpret the user's inner world. Once the camera captures the above image data, the system will immediately start its powerful analysis function. Behind this is a set of complex and sophisticated algorithms that can deeply process and analyze each frame of the image. By comparing tens of thousands of data points and known emotional expression patterns, the system can accurately identify the user's various emotions, whether it is a happy smile, sad tears, or angry frowns. The psychological state recognition system in the prior art has problems such as poor accuracy and poor stability. Summary of the invention

[0003] The present invention provides a psychological state recognition system and method based on facial images and physiological information, which is used to solve the problems of poor accuracy and poor stability of the psychological state recognition system in the prior art. The technical solutions adopted are as follows:

[0004] A psychological state recognition system based on facial images and physiological information, comprising:

[0005] Cabin;

[0006] Various types of image acquisition devices are built into the cabin and are used to acquire images of users entering the cabin to form image data;

[0007] Various types of biosensor groups are built into the cabin to collect physiological information parameters of the user and form physiological index data;

[0008] The data processing platform is connected to the image acquisition device and the biosensor group for data communication via wireless communication, and is used to obtain the image data and physiological index data, and perform data analysis and user psychological state recognition.

[0009] Furthermore, the multiple types of image acquisition devices include infrared cameras, optical cameras and high-definition cameras; the biosensor group includes electrocardiogram sensors, blood pressure sensors, galvanic skin response sensors and respiratory sensors.

[0010] Furthermore, the data processing platform includes:

[0011] The data transmission frequency setting module is used to set the image data transmission frequency and the physiological index data transmission frequency according to the physiological index data of the user in a preset trial monitoring time period when there is a user to be identified in the cabin;

[0012] A standby communication channel startup constraint condition acquisition module is used to set the standby communication channel startup constraint condition according to the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set;

[0013] A psychological state recognition module is used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user and recognize the psychological state after the backup communication channel startup constraint conditions are set;

[0014] The standby communication channel startup determination module is used to monitor the data transmission operation parameters in the data transmission process of image data and physiological index data in real time, and use the data transmission operation parameters and the standby communication channel startup constraint conditions to determine whether to start the standby communication channel.

[0015] Furthermore, the data transmission frequency setting module includes:

[0016] The personnel in-cabin identification module is used to activate various types of image acquisition devices in the cabin and determine whether there is a user to be identified in the cabin through the image data sent back by the image acquisition device;

[0017] The physiological index data acquisition module is used to control the biosensor group to collect the user's physiological index data in real time according to the preset initial physiological index data acquisition frequency within the preset trial monitoring time period when there is a user to be identified in the cabin; wherein the physiological index data includes the rhythm of the heartbeat and the proportion of abnormal heart rate, the blood pressure change rate, the skin resistance and the respiratory rate information;

[0018] The frequency setting acquisition module is used to set the image data sending frequency and the physiological indicator data sending frequency according to the physiological indicator data of the user within the preset trial monitoring time period.

[0019] Furthermore, the frequency setting acquisition module includes:

[0020] A first information retrieval module is used to retrieve the heartbeat rhythm, the proportion of abnormal heart rate and the blood pressure change rate contained in the physiological index data;

[0021] The first frequency adjustment parameter acquisition module is used to obtain the first frequency adjustment parameter by using the heartbeat rhythm, the proportion of abnormal heart rate and the blood pressure change rate; wherein the first frequency adjustment parameter is obtained by the following formula:

[0022]

[0023]

[0024]

[0025] Among them, ξ 01 represents the first frequency adjustment parameter; m represents the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; δ represents the floating coefficient corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; B max and B min Respectively represent the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; B p It represents the average value of the heartbeat rhythm and the proportion of abnormal heart rate; t The ratio between the difference between the systolic pressure value in the blood pressure corresponding to the time when the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate appear and the systolic pressure value of the initial blood pressure detected by the user; p represents the systolic blood pressure change rate in the blood pressure change rate during the preset monitoring period; λ represents the time parameter coefficient; B p Indicates the average value of the heartbeat rhythm and the proportion of abnormal heart rate in the preset test monitoring period; B i represents the value of the rhythm of the heartbeat and the proportion of abnormal heart rate corresponding to the i-th first unit time; ΔT represents the time interval between the maximum and minimum values ​​of the rhythm of the heartbeat and the proportion of abnormal heart rate; T represents the time length value corresponding to the preset trial monitoring time period; e represents a preset minimum constant used to prevent the denominator from being 0;

[0026] A second information retrieval module, used to retrieve the respiratory rate information and skin resistance included in the physiological index data;

[0027] The second frequency adjustment parameter acquisition module is used to obtain the second frequency adjustment parameter using the respiratory frequency information and skin resistance; wherein the second frequency adjustment parameter is obtained by the following formula:

[0028]

[0029]

[0030] Among them, ξ 02represents the second frequency adjustment parameter; k represents the number of second unit time contained in the preset trial monitoring time period, and the value of the second unit time is 1min; H t Indicates the difference between the maximum and minimum values ​​of the respiratory frequency corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; H f Indicates the respiratory rate floating value of the preset trial monitoring period; H i represents the respiratory frequency corresponding to the i-th second unit time; R max and R min The values ​​of skin resistance corresponding to the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; R m Indicates the maximum value of skin resistance during the preset test monitoring period;

[0031] a parameter difference acquisition module, configured to compare the first frequency adjustment parameter with the second frequency adjustment parameter to acquire a difference between the first frequency adjustment parameter and the second frequency adjustment parameter;

[0032] The first sending frequency setting execution module is used to obtain the image data sending frequency and the physiological index data sending frequency by using the first frequency setting model when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is lower than a preset difference threshold; wherein the model structure of the first frequency setting model is as follows:

[0033]

[0034]

[0035] Among them, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological index data; F s0 Indicates the initial frequency of sending physiological index data; ξ y Represents the preset difference threshold; ξ c represents the difference between the first frequency adjustment parameter and the second frequency adjustment parameter;

[0036] The second sending frequency setting execution module is used to obtain the image data sending frequency and the physiological index data sending frequency by using the second frequency setting model when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is not less than a preset difference threshold; wherein the model structure of the second frequency setting model is as follows:

[0037]

[0038]

[0039] Among them, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological index data; F s0 Indicates the initial frequency of sending physiological index data; ξ y Represents the preset difference threshold; ξ c Indicates the difference between the first frequency adjustment parameter and the second frequency adjustment parameter.

[0040] Furthermore, the backup communication channel startup constraint condition acquisition module includes:

[0041] A data transmission operation parameter acquisition module is used to extract the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set; wherein the data transmission operation parameters include data transmission rate, bit error rate, data delay data and throughput;

[0042] The first constraint condition parameter acquisition module is used to acquire the first constraint condition parameter according to the data transmission rate, bit error rate and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the first constraint condition parameter is acquired by the following formula:

[0043]

[0044] Among them, α 01 represents the first constraint condition parameter; m represents the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; b i represents the data transmission rate of the first unit time of the ith time; b e Indicates the standard data transmission rate of the communication channel; p i represents the bit error rate of the first unit time of the i-th time; p cmax Indicates the bit error rate corresponding to the maximum throughput within the preset test monitoring time period; p max Indicates the maximum value of the bit error rate within the preset test monitoring period; e indicates a preset minimum constant, which is used to prevent the denominator from being 0;

[0045] The second constraint condition parameter acquisition module is used to acquire the second constraint condition parameter according to the data delay data and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the second constraint condition parameter is acquired by the following formula:

[0046]

[0047] Among them, α 02 represents the second constraint parameter; T i represents the average data transmission delay corresponding to the i-th unit time; T y Indicates the maximum delay allowed for data transmission; C i represents the throughput of the first unit time of the ith time; C e Indicates the preset throughput reference value; p ti represents the bit error rate in the data corresponding to the data transmission delay of the i-th first unit time;

[0048] A communication quality reference retrieving module, used to retrieve a preset communication quality reference value;

[0049] A constraint condition acquisition module, used to set the upper limit value and the lower limit value of the standby communication channel startup constraint condition by using the first constraint condition parameter and the first constraint condition parameter in combination with a preset communication quality reference value; wherein the upper limit value and the lower limit value of the communication channel startup constraint condition are obtained by the following formula:

[0050]

[0051]

[0052] Among them, S up and S down They respectively represent the upper limit and lower limit of the activation constraint condition of the standby communication channel; S0 represents a preset communication quality reference value; Ψ represents a compensation coefficient; and the Ψ is obtained by the following formula:

[0053]

[0054] Among them, C b Indicates the percentage value corresponding to the communication channel saturation.

[0055] Furthermore, the mental state recognition module includes:

[0056] An image acquisition control module, used to control the plurality of types of image acquisition devices to acquire real-time facial image data of a user, and to send the facial image data of the user to a data processing platform according to a preset image data sending frequency;

[0057] A physiological index data acquisition control module, used to use the biosensor group to collect the user's physiological index data in real time, and send the physiological index data to a data processing platform according to a preset physiological index data sending frequency;

[0058] The psychological state information acquisition module is used to input the target image data and target physiological indicator data into a trained psychological state recognition model, and identify the user's current psychological state through the trained psychological state recognition model to obtain the user's emotional state data, mental stress level data and mental state data.

[0059] Furthermore, the mental state recognition module includes:

[0060] A data acquisition control module, used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user after the backup communication channel startup constraint conditions are set;

[0061] A data input module, used for inputting the image data and physiological index data into the trained mental state recognition model;

[0062] The psychological state recognition module is used to recognize the user's current psychological state through the trained psychological state recognition model, and obtain the user's emotional state data, mental stress level data and mental state data.

[0063] Furthermore, the standby communication channel startup determination module includes:

[0064] A real-time monitoring module, used for real-time monitoring of data transmission operating parameters during the data transmission process of image data and physiological index data, wherein the data transmission operating parameters include data transmission rate, bit error rate, data delay data and throughput;

[0065] The communication transmission evaluation parameter acquisition module is used to obtain the communication transmission evaluation parameter using the data transmission operation parameter; wherein the communication transmission evaluation parameter is obtained by the following formula:

[0066]

[0067] Where S represents the communication transmission evaluation parameter; n represents the number of first unit time contained in the current communication duration; b j represents the data transmission rate of the jth first unit time; b e Indicates the standard data transmission rate of the communication channel; p j represents the bit error rate of the jth first unit time; p tj represents the delay rate of the jth first unit time; C j represents the communication capacity occupancy rate of the jth first unit time; p sj Indicates the ratio of the amount of data with bit errors in the delayed data of the j-th first unit time to the total amount of data transmission of the j-th first unit time;

[0068] The backup communication channel activation module is used to activate the backup communication channel when the communication transmission evaluation parameter exceeds the constraint range corresponding to the communication channel activation constraint condition.

[0069] A method for identifying a psychological state based on facial images and physiological information, the method comprising:

[0070] When there is a user to be identified in the cabin, the image data sending frequency and the physiological index data sending frequency are set according to the user's physiological index data in a preset trial monitoring time period;

[0071] After the image data transmission frequency and the physiological index data transmission frequency are set, setting the backup communication channel startup constraint conditions according to the data transmission operation parameters of the physiological index data in the trial monitoring time period;

[0072] After the backup communication channel startup constraint conditions are set, various types of image acquisition devices and biosensor groups are controlled to collect image data and physiological index data of the user and recognize the psychological state;

[0073] The data transmission operation parameters in the data transmission process of the image data and the physiological index data are monitored in real time, and the data transmission operation parameters and the activation constraint conditions of the backup communication channel are used to determine whether to activate the backup communication channel.

[0074] Beneficial effects of the present invention:

[0075] The present invention proposes a psychological state recognition system and method based on facial images and physiological information. Through various types of image acquisition devices and biosensor groups, the system can comprehensively and accurately collect the user's facial images and physiological information, providing a rich data basis for psychological state recognition. The system can dynamically adjust the sending frequency of image data and physiological indicator data according to the user's physiological indicator data to ensure the efficiency and stability of data transmission. At the same time, the setting of the backup communication channel further enhances the reliability of data transmission. Based on the comprehensive data collected, the system can accurately identify the user's psychological state and provide strong support for mental health assessment, disease warning, etc. By real-time monitoring of data transmission operating parameters and determining whether it is necessary to start the backup communication channel, the system can respond in time when there is a problem with data transmission, ensuring the continuity and accuracy of psychological state recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a system block diagram of the system of the present invention;

[0077] Figure 2 is a system block diagram of the data processing platform of the present invention;

[0078] Figure 3 The present invention is a flowchart of the method. DETAILED DESCRIPTION

[0079] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0080] The embodiment of the present invention proposes a psychological state recognition system based on facial images and physiological information, such as Figure 1 As shown, the psychological state recognition system based on facial images and physiological information includes a cabin, various types of image acquisition devices, a biosensor group and a data processing platform; various types of image acquisition devices are built into the cabin, and are used to capture images of users entering the cabin to form image data; various types of biosensor groups are built into the cabin, and are used to collect physiological information parameters of users to form physiological indicator data; the data processing platform is connected to the image acquisition device and the biosensor group for data communication via wireless communication, and is used to obtain the image data and physiological indicator data, and perform data analysis and user psychological state recognition.

[0081] Among them, the multiple types of image acquisition devices include infrared cameras, optical cameras and high-definition cameras; the biosensor group includes electrocardiogram sensors, blood pressure sensors, skin electrical response sensors and breathing sensors.

[0082] And, if Figure 2 As shown, the data processing platform includes:

[0083] A data transmission frequency setting module is used to set the image data transmission frequency and the physiological index data transmission frequency according to the physiological index data of the user in a preset trial monitoring time period when there is a user to be identified in the cabin;

[0084] A standby communication channel startup constraint condition acquisition module is used to set the standby communication channel startup constraint condition according to the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set;

[0085] A psychological state recognition module is used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user and recognize the psychological state after the backup communication channel startup constraint conditions are set;

[0086] The standby communication channel startup determination module is used to monitor the data transmission operation parameters in the data transmission process of image data and physiological index data in real time, and use the data transmission operation parameters and the standby communication channel startup constraint conditions to determine whether to start the standby communication channel.

[0087] The working principle of the above technical solution is: the system collects the user's image data and physiological index data through various types of image acquisition devices (including infrared cameras, optical cameras and high-definition cameras) and biosensor groups (including electrocardiogram sensors, blood pressure sensors, skin galvanic response sensors and breathing sensors) set in the cabin. Then, the data is sent to the data processing platform through the set communication channel. The data transmission frequency setting module will set the transmission frequency of image data and physiological index data according to the user's physiological index data during the trial monitoring period.

[0088] At the same time, the system also has a backup communication channel startup constraint condition acquisition module, which sets the startup constraint conditions of the backup communication channel according to the data transmission operation parameters of the physiological indicator data during the trial monitoring period. After the startup constraint conditions of the backup communication channel are set, the psychological state recognition module controls the image acquisition device and the biosensor group to collect image data and physiological indicator data of the user, and perform psychological state recognition. Finally, the backup communication channel startup determination module monitors the data transmission operation parameters during the data transmission process in real time, and uses the above parameters and the startup constraint conditions of the backup communication channel to determine whether the backup communication channel needs to be started.

[0089] The effect of the above technical solution is: through various types of image acquisition devices and biosensor groups, the system can comprehensively and accurately collect the user's facial images and physiological information, providing a rich data basis for psychological state recognition. The system can dynamically adjust the sending frequency of image data and physiological indicator data according to the user's physiological indicator data to ensure the efficiency and stability of data transmission. At the same time, the setting of the backup communication channel further enhances the reliability of data transmission. Based on the comprehensive data collected, the system can accurately identify the user's psychological state and provide strong support for mental health assessment, disease warning, etc. By real-time monitoring of data transmission operating parameters and determining whether it is necessary to start the backup communication channel, the system can respond in time when there is a problem with data transmission to ensure the continuity and accuracy of psychological state recognition.

[0090] In one embodiment of the present invention, the data transmission frequency setting module includes:

[0091] The personnel in-cabin identification module is used to activate various types of image acquisition devices in the cabin and determine whether there is a user to be identified in the cabin through the image data sent back by the image acquisition device;

[0092] The physiological index data acquisition module is used to control the biosensor group to collect the user's physiological index data in real time according to the preset initial physiological index data acquisition frequency within the preset trial monitoring time period when there is a user to be identified in the cabin; wherein the physiological index data includes the rhythm of the heartbeat and the proportion of abnormal heart rate, the blood pressure change rate, the skin resistance and the respiratory rate information;

[0093] The frequency setting acquisition module is used to set the image data sending frequency and the physiological indicator data sending frequency according to the physiological indicator data of the user within the preset trial monitoring time period.

[0094] The working principle of the above technical solution is as follows: First, the personnel in the cabin identification module will start various types of image acquisition devices in the cabin, and judge whether there is a user to be identified in the current cabin through the image data sent back by the above devices. This is to ensure that the system only performs subsequent data collection and processing when there is a user present. When it is confirmed that there is a user to be identified in the cabin, the physiological indicator data acquisition module will collect the user's physiological indicator data in real time through the biosensor group within a preset trial monitoring time period according to the preset initial physiological indicator data acquisition frequency. The above data includes key information such as heartbeat rhythm, abnormal heart rate ratio, blood pressure change rate, skin resistance and respiratory rate. The above data is crucial for subsequent judgment of the user's psychological state. After collecting the user's physiological indicator data, the frequency setting acquisition module will set the image data transmission frequency and the physiological indicator data transmission frequency according to the above data. The above setting process is to optimize the data transmission efficiency, ensure that unnecessary data transmission is reduced without affecting the data quality, thereby reducing the communication burden of the system.

[0095] The effect of the above technical solution is that the system can intelligently identify whether there is a user in the cabin, so as to decide whether to start the subsequent data collection and processing process, which improves the intelligence of the system and the user experience. By collecting a variety of physiological indicator data of the user in real time, the system can have a more comprehensive understanding of the user's physiological state and provide a more accurate data basis for psychological state identification. By dynamically setting the data transmission frequency according to the user's physiological indicator data, the system can effectively reduce communication costs and improve the overall efficiency of the system while ensuring the quality of data transmission. Since each person's physiological reactions and psychological states are different, the system can provide users with more accurate and personalized services by personalizing the data transmission frequency.

[0096] In one embodiment of the present invention, a frequency setting acquisition module includes:

[0097] A first information retrieval module is used to retrieve the heartbeat rhythm, the proportion of abnormal heart rate and the blood pressure change rate contained in the physiological index data;

[0098] The first frequency adjustment parameter acquisition module is used to obtain the first frequency adjustment parameter by using the heartbeat rhythm, the proportion of abnormal heart rate and the blood pressure change rate; wherein the first frequency adjustment parameter is obtained by the following formula:

[0099]

[0100]

[0101]

[0102] Among them, ξ 01 represents the first frequency adjustment parameter; m represents the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; δ represents the floating coefficient corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; B max and B min Respectively represent the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; B p It represents the average value of the heartbeat rhythm and the proportion of abnormal heart rate; t The ratio between the difference between the systolic pressure value in the blood pressure corresponding to the time when the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate appear and the systolic pressure value of the initial blood pressure detected by the user; p represents the systolic blood pressure change rate in the blood pressure change rate during the preset monitoring period; λ represents the time parameter coefficient; B p Indicates the average value of the heartbeat rhythm and the proportion of abnormal heart rate in the preset test monitoring period; B i represents the value of the rhythm of the heartbeat and the proportion of abnormal heart rate corresponding to the i-th first unit time; ΔT represents the time interval between the maximum and minimum values ​​of the rhythm of the heartbeat and the proportion of abnormal heart rate; T represents the time length value corresponding to the preset trial monitoring time period; e represents a preset minimum constant used to prevent the denominator from being 0;

[0103] A second information retrieval module, used to retrieve the respiratory rate information and skin resistance included in the physiological index data;

[0104] The second frequency adjustment parameter acquisition module is used to obtain the second frequency adjustment parameter using the respiratory frequency information and skin resistance; wherein the second frequency adjustment parameter is obtained by the following formula:

[0105]

[0106]

[0107] Among them, ξ 01represents the first frequency adjustment parameter; m represents the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; δ represents the floating coefficient corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; B max and B min Respectively represent the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; B p It represents the average value of the heartbeat rhythm and the proportion of abnormal heart rate; t Represents the ratio between the difference between the systolic pressure values ​​in the blood pressure corresponding to the maximum and minimum values ​​of the proportion of the heartbeat rhythm and heart rate abnormality and the systolic pressure value of the initial blood pressure detected by the user (the difference between the systolic pressure values ​​in the blood pressure corresponding to the maximum and minimum values ​​of the proportion of the heartbeat rhythm and heart rate abnormality is the difference between the systolic pressure of the maximum value of the proportion of the heartbeat rhythm and heart rate abnormality and the systolic pressure of the minimum value of the proportion of the heartbeat rhythm and heart rate abnormality); f p represents the systolic blood pressure change rate in the blood pressure change rate during the preset monitoring period; λ represents the time parameter coefficient; B p Indicates the average value of the heartbeat rhythm and the proportion of abnormal heart rate in the preset test monitoring period; B i represents the value of the rhythm of the heartbeat and the proportion of abnormal heart rate corresponding to the i-th first unit time; ΔT represents the time interval between the maximum and minimum values ​​of the rhythm of the heartbeat and the proportion of abnormal heart rate; T represents the time length value corresponding to the preset trial monitoring time period; e represents a preset minimum constant used to prevent the denominator from being 0;

[0108] a parameter difference acquisition module, configured to compare the first frequency adjustment parameter with the second frequency adjustment parameter to acquire a difference between the first frequency adjustment parameter and the second frequency adjustment parameter;

[0109] The first sending frequency setting execution module is used to obtain the image data sending frequency and the physiological index data sending frequency by using the first frequency setting model when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is lower than a preset difference threshold; wherein the model structure of the first frequency setting model is as follows:

[0110]

[0111]

[0112] Among them, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological index data; F s0Indicates the initial frequency of sending physiological index data; ξ y Represents the preset difference threshold; ξ c represents the difference between the first frequency adjustment parameter and the second frequency adjustment parameter;

[0113] The second sending frequency setting execution module is used to obtain the image data sending frequency and the physiological index data sending frequency by using the second frequency setting model when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is not less than a preset difference threshold; wherein the model structure of the second frequency setting model is as follows:

[0114]

[0115]

[0116] Among them, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological index data; F s0 Indicates the initial frequency of sending physiological index data; ξ y Represents the preset difference threshold; ξ c Indicates the difference between the first frequency adjustment parameter and the second frequency adjustment parameter.

[0117] The working principle of the above technical solution is as follows: the system first retrieves the heart rhythm, abnormal heart rate ratio and blood pressure change rate in the user's physiological index data through the first information retrieval module. Then, the first frequency adjustment parameter acquisition module uses the above data in combination with a specific formula to calculate the first frequency adjustment parameter. The above parameters reflect the changes in the user's heartbeat and blood pressure. Then, the system retrieves the user's respiratory frequency information and skin resistance through the second information retrieval module. Then, the second frequency adjustment parameter acquisition module uses the above data in combination with another specific formula to calculate the second frequency adjustment parameter. The above parameters reflect the changes in the user's breathing and skin resistance. The parameter difference acquisition module compares the first frequency adjustment parameter with the second frequency adjustment parameter to obtain the difference between them. The above difference reflects the consistency of changes between different physiological indicators of the user. According to the comparison result of the difference between the first frequency adjustment parameter and the second frequency adjustment parameter and the preset difference threshold, the system selects to use the first frequency setting model or the second frequency setting model to obtain the image data sending frequency and the physiological index data sending frequency. If the difference is lower than the difference threshold, it means that the changes in the user's physiological indicators are relatively consistent, and the system uses the first frequency setting model; if the difference is not lower than the difference threshold, it means that there are large differences in the changes in the user's physiological indicators, and the system uses the second frequency setting model.

[0118] The effect of the above technical solution is that the system can dynamically adjust the data transmission frequency according to the actual physiological indicators of the user, which not only improves the efficiency of data transmission, but also better adapts to the changes in the physiological states of different users. By comprehensively considering multiple physiological indicators such as heart rate, blood pressure, respiration and skin resistance, the system can more comprehensively evaluate the user's physiological state and provide a more accurate data basis for subsequent psychological state identification. The system can select different frequency setting models according to the degree of consistency between physiological indicators, which enhances the flexibility and adaptability of the system, enabling it to maintain efficient data transmission in different situations. By dynamically adjusting the data transmission frequency, the system can reduce communication costs and optimize resource utilization while ensuring the quality of data transmission.

[0119] In one embodiment of the present invention, the standby communication channel startup constraint condition acquisition module includes:

[0120] A data transmission operation parameter acquisition module is used to extract the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set; wherein the data transmission operation parameters include data transmission rate, bit error rate, data delay data and throughput;

[0121] The first constraint condition parameter acquisition module is used to acquire the first constraint condition parameter according to the data transmission rate, bit error rate and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the first constraint condition parameter is acquired by the following formula:

[0122]

[0123] Among them, α 01 represents the first constraint condition parameter; m represents the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; b i represents the data transmission rate of the first unit time of the ith time; b e Indicates the standard data transmission rate of the communication channel; p i represents the bit error rate of the first unit time of the i-th time; p cmax Indicates the bit error rate corresponding to the maximum throughput within the preset test monitoring time period; p max Indicates the maximum value of the bit error rate within the preset test monitoring period; e indicates a preset minimum constant, which is used to prevent the denominator from being 0;

[0124] The second constraint condition parameter acquisition module is used to acquire the second constraint condition parameter according to the data delay data and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the second constraint condition parameter is acquired by the following formula:

[0125]

[0126] Among them, α 02 represents the second constraint parameter; T i represents the average data transmission delay corresponding to the i-th unit time; T y Indicates the maximum delay allowed for data transmission; C i represents the throughput of the first unit time of the ith time; C e Indicates the preset throughput reference value; p ti represents the bit error rate in the data corresponding to the data transmission delay of the i-th first unit time;

[0127] A communication quality reference retrieving module, used to retrieve a preset communication quality reference value;

[0128] A constraint condition acquisition module, used to set the upper limit value and the lower limit value of the standby communication channel startup constraint condition by using the first constraint condition parameter and the first constraint condition parameter in combination with a preset communication quality reference value; wherein the upper limit value and the lower limit value of the communication channel startup constraint condition are obtained by the following formula:

[0129]

[0130]

[0131] Among them, S up and S down They respectively represent the upper limit and lower limit of the activation constraint condition of the standby communication channel; S0 represents a preset communication quality reference value; Ψ represents a compensation coefficient; and the Ψ is obtained by the following formula:

[0132]

[0133] Among them, C b Indicates the percentage value corresponding to the communication channel saturation.

[0134] The working principle of the above technical solution is as follows: after the image data transmission frequency and the physiological index data transmission frequency are set, the system extracts the data transmission operation parameters of the physiological index data within the trial monitoring period through the data transmission operation parameter acquisition module. The above parameters include data transmission rate, bit error rate, data delay and throughput, which are important indicators for evaluating the performance of the communication channel. The first constraint condition parameter acquisition module calculates the first constraint condition parameter according to the data transmission rate, bit error rate and throughput. The above parameters reflect the throughput performance of the communication channel under the given data transmission rate and bit error rate. The second constraint condition parameter acquisition module calculates the second constraint condition parameter according to the data delay and throughput. The above parameters take into account the impact of data transmission delay on throughput and are important indicators for measuring the real-time performance of the communication channel. The communication quality reference retrieval module retrieves the preset communication quality reference value, which is used as a benchmark for the subsequent setting of the backup communication channel startup constraint. The constraint condition acquisition module sets the upper limit and lower limit of the backup communication channel startup constraint in combination with the first constraint condition parameter, the second constraint condition parameter and the preset communication quality reference value. The above constraints are used to determine when to start the backup communication channel to ensure the quality and stability of data transmission. The compensation coefficient Ψ is calculated based on the percentage value corresponding to the communication channel saturation, and is used to adjust the upper and lower limits of the startup constraint conditions of the backup communication channel to adapt to different communication environments and requirements.

[0135] The effect of the above technical solution is: by extracting data transmission operation parameters in real time, the system can dynamically evaluate the performance status of the current communication channel and provide accurate decision-making basis for the startup of the backup communication channel. Combining multiple data transmission operation parameters and preset communication quality reference values, the system can flexibly set the constraints for the startup of the backup communication channel to adapt to different communication needs and scenarios. When the performance of the main communication channel degrades or fails, the system can switch to the backup channel in time according to the startup constraints of the backup communication channel, thereby ensuring the quality and stability of data transmission. Through reasonable constraint setting, the system can optimize the utilization of communication resources and reduce communication costs while ensuring the quality of data transmission.

[0136] In one embodiment of the present invention, the mental state recognition module includes:

[0137] An image acquisition control module, used to control the plurality of types of image acquisition devices to acquire real-time facial image data of a user, and to send the facial image data of the user to a data processing platform according to a preset image data sending frequency;

[0138] A physiological index data acquisition control module, used to use the biosensor group to collect the user's physiological index data in real time, and send the physiological index data to a data processing platform according to a preset physiological index data sending frequency;

[0139] The psychological state information acquisition module is used to input the target image data and target physiological indicator data into a trained psychological state recognition model, and identify the user's current psychological state through the trained psychological state recognition model to obtain the user's emotional state data, mental stress level data and mental state data.

[0140] The mental state recognition model adopts a neural network model, and the neural network model generally includes:

[0141] Input layer: Receives raw image data and physiological indicator data. For image data, a convolutional layer may be used to extract features; for physiological indicator data, a fully connected layer or a specific feature extraction method may be used.

[0142] Feature extraction layer: This layer is responsible for extracting meaningful features from the raw data for use by subsequent classification or regression layers. In deep learning, this is usually done automatically through convolutional layers, pooling layers, etc.

[0143] Classification / Regression Layer: Classify or regress the user's mental state based on the extracted features. For classification tasks, the softmax function is usually used to output the probability of each category; for regression tasks, the predicted value is directly output.

[0144] Output layer: Outputs the final psychological state recognition results, including emotional state data, mental stress level data, and mental state data.

[0145] The working principle of the above technical solution is as follows: the image acquisition control module is responsible for controlling various types of image acquisition devices (such as cameras, etc.) to capture the user's facial image data in real time. The above image data will then be transmitted to the data processing platform according to the preset image data transmission frequency. This is done to ensure the real-time and accuracy of the data while reducing the burden of data transmission. The physiological index data acquisition control module collects the user's physiological index data, such as heart rate, skin resistance, etc., in real time through a biosensor group (such as an electrocardiogram monitor, a skin galvanic response sensor, etc.). The above data will also be sent to the data processing platform according to the preset physiological index data transmission frequency. The above real-time physiological data acquisition technical solution helps to more accurately analyze the user's psychological state. The psychological state information acquisition module receives image data and physiological index data from the data processing platform. The above data is input into a trained psychological state recognition model. The model is trained with a large amount of data and can recognize and analyze the user's psychological state. Through the calculation and processing of the model, the system can recognize the user's emotional state (such as happiness, sadness, anger, etc.), mental stress level and mental state (such as fatigue, concentration, etc.).

[0146] The effect of the above technical solution is: by collecting images and physiological data in real time and sending them to the data processing platform immediately, the above technical solution ensures the real-time and accuracy of the data, and provides a reliable data basis for psychological state recognition. Combined with facial images and physiological indicator data, this technology can comprehensively analyze the user's psychological state from multiple dimensions, improving the accuracy and depth of recognition. Using the trained psychological state recognition model, the system can perform intelligent analysis based on the individual differences of each user and provide personalized psychological state recognition results. The above psychological state recognition technology can be applied to multiple fields, such as mental health monitoring, human-computer interaction optimization, advertising and marketing effect evaluation, etc. The above technical solution helps to improve user experience and quality of life.

[0147] In one embodiment of the present invention, the mental state recognition module includes:

[0148] A data acquisition control module, used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user after the backup communication channel startup constraint conditions are set;

[0149] A data input module, used for inputting the image data and physiological index data into the trained mental state recognition model;

[0150] The psychological state recognition module is used to recognize the user's current psychological state through the trained psychological state recognition model, and obtain the user's emotional state data, mental stress level data and mental state data.

[0151] The working principle of the above technical solution is as follows: the image acquisition control module is responsible for controlling various types of image acquisition devices (such as cameras, etc.) to capture the user's facial image data in real time. The above image data will then be transmitted to the data processing platform according to the preset image data transmission frequency. This is done to ensure the real-time and accuracy of the data while reducing the burden of data transmission.

[0152] The physiological index data acquisition control module collects the user's physiological index data in real time, such as heart rate, skin resistance, etc., through a biosensor group (such as an electrocardiogram monitor, a skin galvanic response sensor, etc.). The above data will also be sent to the data processing platform according to the preset physiological index data sending frequency. The above real-time physiological data acquisition technical solution helps to more accurately analyze the user's psychological state.

[0153] The psychological state information acquisition module receives image data and physiological index data from the data processing platform. The above data is input into a trained psychological state recognition model. After being trained with a large amount of data, the model can recognize and analyze the user's psychological state. Through the calculation and processing of the model, the system can recognize the user's emotional state (such as happiness, sadness, anger, etc.), mental stress level and mental state (such as fatigue, concentration, etc.).

[0154] The effect of the above technical solution is: by collecting images and physiological data in real time and sending them to the data processing platform immediately, the above technical solution ensures the real-time and accuracy of the data, and provides a reliable data basis for psychological state recognition. Combined with facial images and physiological indicator data, this technology can comprehensively analyze the user's psychological state from multiple dimensions, improving the accuracy and depth of recognition. Using the trained psychological state recognition model, the system can perform intelligent analysis based on the individual differences of each user and provide personalized psychological state recognition results. The above psychological state recognition technology can be applied to multiple fields, such as mental health monitoring, human-computer interaction optimization, advertising and marketing effect evaluation, etc. The above technical solution helps to improve user experience and quality of life.

[0155] In one embodiment of the present invention, the standby communication channel startup determination module includes:

[0156] A real-time monitoring module, used for real-time monitoring of data transmission operating parameters during the data transmission process of image data and physiological index data, wherein the data transmission operating parameters include data transmission rate, bit error rate, data delay data and throughput;

[0157] The communication transmission evaluation parameter acquisition module is used to obtain the communication transmission evaluation parameter using the data transmission operation parameter; wherein the communication transmission evaluation parameter is obtained by the following formula:

[0158]

[0159] Where S represents the communication transmission evaluation parameter; n represents the number of first unit time contained in the current communication duration; b j represents the data transmission rate of the jth first unit time; b e Indicates the standard data transmission rate of the communication channel; p j represents the bit error rate of the jth first unit time; p tj represents the delay rate of the jth first unit time; C j represents the communication capacity occupancy rate of the jth first unit time; p sj represents the ratio of the amount of data with bit errors in the delayed data of the j-th first unit time to the total amount of data transmission of the j-th first unit time;

[0160] The backup communication channel activation module is used to activate the backup communication channel when the communication transmission evaluation parameter exceeds the constraint range corresponding to the communication channel activation constraint condition.

[0161] Among them, when the communication transmission evaluation parameter is lower than the lower limit value of the constraint range corresponding to the communication channel startup constraint condition, it indicates that the current data transmission quality is poor; when the communication transmission evaluation parameter exceeds the upper limit value of the constraint range corresponding to the communication channel startup constraint condition, it indicates that the current data transmission quality load is large.

[0162] The working principle of the above technical solution is: the real-time monitoring module will continuously monitor various data transmission operating parameters of image data and physiological index data during the transmission process, including data transmission rate, bit error rate, data delay and throughput. The above parameters are key indicators for evaluating data transmission quality.

[0163] The communication transmission evaluation parameter acquisition module uses the real-time monitored data transmission operation parameters to calculate the communication transmission evaluation parameters through a specific formula. The above formula comprehensively considers multiple factors such as data transmission rate, bit error rate, delay rate and communication capacity occupancy rate, as well as the proportion of bit errors in delay data, to obtain a comprehensive communication transmission evaluation parameter.

[0164] The backup communication channel activation module will compare the communication transmission evaluation parameters with the preset communication channel activation constraints. If the communication transmission evaluation parameters exceed the range of the constraints (whether below the lower limit or above the upper limit), the system will determine that the backup communication channel needs to be activated. Below the lower limit usually means that the data transmission quality is poor, and there may be a high bit error rate or data delay; while above the upper limit may indicate that the current communication load is too large and there is a potential risk of data congestion.

[0165] The effect of the above technical solution is: through the real-time monitoring module, the system can grasp the status of data transmission in real time and discover potential problems in time. By setting the communication channel startup constraint conditions, the system can start the backup communication channel in time before the data transmission quality is significantly reduced or the communication load is too large, thereby preventing data transmission interruption or serious quality degradation. The formulas and constraints in this technical solution can be adjusted according to actual needs to adapt to different application scenarios and requirements. Comprehensive consideration of multiple data transmission operating parameters to evaluate communication quality makes the judgment result more comprehensive and reliable. The above technical solution can start the backup communication channel in time when necessary by real-time monitoring and comprehensive evaluation of multiple key indicators in the data transmission process, thereby ensuring the stability and reliability of data transmission.

[0166] The embodiment of the present invention proposes a method for identifying a psychological state based on facial images and physiological information. Figure 3 As shown, the psychological state recognition method based on facial images and physiological information includes:

[0167] S1. When there is a user to be identified in the cabin, the image data sending frequency and the physiological index data sending frequency are set according to the user's physiological index data within a preset trial monitoring time period;

[0168] S2. After the image data transmission frequency and the physiological index data transmission frequency are set, setting the backup communication channel startup constraint conditions according to the data transmission operation parameters of the physiological index data within the trial monitoring time period;

[0169] S3, after the backup communication channel startup constraint conditions are set, control various types of image acquisition devices and biosensor groups to collect image data and physiological index data and identify psychological states of the user;

[0170] S4. Real-time monitoring of data transmission operation parameters during the data transmission of image data and physiological index data, and determining whether to start the backup communication channel by using the data transmission operation parameters and the backup communication channel start constraint conditions.

[0171] The working principle of the above technical solution is: the system collects the user's image data and physiological index data through various types of image acquisition devices (including infrared cameras, optical cameras and high-definition cameras) and biosensor groups (including electrocardiogram sensors, blood pressure sensors, skin galvanic response sensors and breathing sensors) set in the cabin. Then, the data is sent to the data processing platform through the set communication channel. The data transmission frequency setting module will set the transmission frequency of image data and physiological index data according to the user's physiological index data during the trial monitoring period.

[0172] At the same time, the system also has a backup communication channel startup constraint condition acquisition module, which sets the startup constraint conditions of the backup communication channel according to the data transmission operation parameters of the physiological indicator data during the trial monitoring period. After the startup constraint conditions of the backup communication channel are set, the psychological state recognition module controls the image acquisition device and the biosensor group to collect image data and physiological indicator data of the user, and perform psychological state recognition. Finally, the backup communication channel startup determination module monitors the data transmission operation parameters during the data transmission process in real time, and uses the above parameters and the startup constraint conditions of the backup communication channel to determine whether the backup communication channel needs to be started.

[0173] The effect of the above technical solution is: through various types of image acquisition devices and biosensor groups, the system can comprehensively and accurately collect the user's facial images and physiological information, providing a rich data basis for psychological state recognition. The system can dynamically adjust the sending frequency of image data and physiological indicator data according to the user's physiological indicator data to ensure the efficiency and stability of data transmission. At the same time, the setting of the backup communication channel further enhances the reliability of data transmission. Based on the comprehensive data collected, the system can accurately identify the user's psychological state and provide strong support for mental health assessment, disease warning, etc. By real-time monitoring of data transmission operating parameters and determining whether it is necessary to start the backup communication channel, the system can respond in time when there is a problem with data transmission to ensure the continuity and accuracy of psychological state recognition.

[0174] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if the above-mentioned modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include the above-mentioned modifications and variations.

Claims

1. A psychological state recognition system based on facial images and physiological information, characterized in that: include: Cabin; Various types of image acquisition devices are built into the cabin and are used to acquire images of users entering the cabin to form image data; Various types of biosensor groups are built into the cabin to collect physiological information parameters of the user and form physiological index data; A data processing platform, which is connected to the image acquisition device and the biosensor group through wireless communication to obtain the image data and physiological index data, and to perform data analysis and user psychological state recognition; The data processing platform includes a frequency setting acquisition module, which includes: The first information retrieval module retrieves the heartbeat rhythm, the proportion of abnormal heart rate and the blood pressure change rate contained in the physiological index data; A first frequency adjustment parameter acquisition module, which acquires a first frequency adjustment parameter by using the rhythm of the heartbeat, the proportion of abnormal heart rate and the blood pressure change rate; The second information retrieval module retrieves the respiratory rate information and skin resistance contained in the physiological index data; A second frequency adjustment parameter acquisition module, which uses the respiratory frequency information and skin resistance to acquire the second frequency adjustment parameter; a parameter difference acquisition module, which compares the first frequency adjustment parameter with the second frequency adjustment parameter to obtain a difference between the first frequency adjustment parameter and the second frequency adjustment parameter; A first sending frequency setting execution module, when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is lower than a preset difference threshold, using the first frequency setting model to obtain the image data sending frequency and the physiological index data sending frequency; The second sending frequency setting execution module uses the second frequency setting model to obtain the image data sending frequency and the physiological index data sending frequency when the difference between the first frequency adjustment parameter and the second frequency adjustment parameter is not less than a preset difference threshold.

2. The psychological state recognition system based on facial images and physiological information according to claim 1, characterized in that: The multiple types of image acquisition devices include infrared cameras, optical cameras and high-definition cameras; the biosensor group includes electrocardiogram sensors, blood pressure sensors, skin electrical response sensors and breathing sensors.

3. The psychological state recognition system based on facial images and physiological information according to claim 1, characterized in that: The data processing platform comprises: A data transmission frequency setting module is used to set the image data transmission frequency and the physiological index data transmission frequency according to the physiological index data of the user in a preset trial monitoring time period when there is a user to be identified in the cabin; A standby communication channel startup constraint condition acquisition module is used to set the standby communication channel startup constraint condition according to the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set; A psychological state recognition module is used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user and recognize the psychological state after the backup communication channel startup constraint conditions are set; The standby communication channel startup determination module is used to monitor the data transmission operation parameters in the data transmission process of image data and physiological index data in real time, and use the data transmission operation parameters and the standby communication channel startup constraint conditions to determine whether to start the standby communication channel.

4. The psychological state recognition system based on facial images and physiological information according to claim 3, characterized in that: The data transmission frequency setting module includes: The personnel in-cabin identification module is used to activate various types of image acquisition devices in the cabin and determine whether there is a user to be identified in the cabin through the image data sent back by the image acquisition device; The physiological index data acquisition module is used to control the biosensor group to collect the user's physiological index data in real time according to the preset initial physiological index data acquisition frequency within the preset trial monitoring time period when there is a user to be identified in the cabin; wherein the physiological index data includes the rhythm of the heartbeat and the proportion of abnormal heart rate, the blood pressure change rate, the skin resistance and the respiratory rate information; The frequency setting acquisition module is used to set the image data sending frequency and the physiological indicator data sending frequency according to the physiological indicator data of the user within the preset trial monitoring time period.

5. The psychological state recognition system based on facial images and physiological information according to claim 4, characterized in that: The first frequency adjustment parameter is obtained by the following formula: in, ξ 01 represents a first frequency adjustment parameter; m Indicates the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; δ Indicates the floating coefficient corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; B max and B min Respectively represent the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; f t Indicates the ratio between the difference between the systolic blood pressure values ​​corresponding to the time when the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate appear and the systolic blood pressure value of the initial blood pressure detected by the user; f p Indicates the systolic blood pressure change rate in the blood pressure change rate during the preset trial monitoring time period; λ represents the time parameter coefficient; B p Indicates the average value of the heartbeat rhythm and the proportion of abnormal heart rate in the preset test monitoring period; B i Indicates i The value of the ratio of abnormal heart rate and heart rate corresponding to the first unit time; Δ T The length of the time interval between the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; T Indicates the time length value corresponding to the preset trial monitoring time period; e Indicates a preset minimum constant, used to prevent the denominator from being 0; The second frequency adjustment parameter is obtained by the following formula: in, ξ 02 represents a second frequency adjustment parameter; δ Indicates the floating coefficient corresponding to the rhythm of the heartbeat and the proportion of abnormal heart rate; B max and B min Respectively represent the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; λ represents the time parameter coefficient; B p Indicates the average value of the heartbeat rhythm and the proportion of abnormal heart rate in the preset test monitoring period; e Indicates a preset minimum constant, used to prevent the denominator from being 0; k Indicates the number of second unit time contained in the preset trial monitoring time period, and the value of the second unit time is 1 minute; H t Indicates the difference in respiratory rate corresponding to the maximum and minimum values ​​of the heartbeat rhythm and the proportion of abnormal heart rate; H f Indicates the floating value of respiratory rate during the preset trial monitoring period; H i Indicates i The respiratory rate corresponding to the second unit time; R max and R min The skin resistance values ​​corresponding to the maximum and minimum values ​​of the heartbeat rhythm and the abnormal heart rate ratio respectively; R m Indicates the maximum value of skin resistance during the preset test monitoring period; The model structure of the first frequency setting model is as follows: in, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological indicator data; F s0 Indicates the frequency of sending initial physiological indicator data; ξ y Indicates the preset difference threshold; ξ c represents the difference between the first frequency adjustment parameter and the second frequency adjustment parameter; The model structure of the second frequency setting model is as follows: in, F x Indicates the frequency of image data transmission; F x0 Indicates the initial image data sending frequency; F s Indicates the frequency of sending physiological indicator data; F s0 Indicates the frequency of sending initial physiological indicator data; ξ y Indicates the preset difference threshold; ξ c Indicates the difference between the first frequency adjustment parameter and the second frequency adjustment parameter.

6. The psychological state recognition system based on facial images and physiological information according to claim 3, characterized in that: The standby communication channel startup constraint condition acquisition module includes: A data transmission operation parameter acquisition module is used to extract the data transmission operation parameters of the physiological indicator data within the trial monitoring time period after the image data transmission frequency and the physiological indicator data transmission frequency are set; wherein the data transmission operation parameters include data transmission rate, bit error rate, data delay data and throughput; The first constraint condition parameter acquisition module is used to acquire the first constraint condition parameter according to the data transmission rate, bit error rate and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the first constraint condition parameter is acquired by the following formula: in, α 01 represents the first constraint parameter; m Indicates the number of first unit time contained in the preset trial monitoring time period, and the value of the first unit time is 1s; b i Indicates i The data transmission rate per first unit time; b e Indicates the standard data transmission rate of a communication channel; p i Indicates i The bit error rate per first unit time; p cmax Indicates the bit error rate corresponding to the maximum throughput within the preset test monitoring time period; p max Indicates the maximum bit error rate within the preset test monitoring time period; e Indicates a preset minimum constant, used to prevent the denominator from being 0; The second constraint condition parameter acquisition module is used to acquire the second constraint condition parameter according to the data delay data and throughput included in the data transmission operation parameters of the physiological indicator data within the trial monitoring time period, wherein the second constraint condition parameter is acquired by the following formula: in, α 02 represents the second constraint parameter; T i Indicates i The average data transmission delay corresponding to the unit time; T y Indicates the maximum delay allowed for data transmission; C i Indicates i The throughput per first unit of time; C e Indicates a preset throughput reference value; p ti Indicates i The bit error rate in the data corresponding to the data transmission delay of the first unit time; A communication quality reference retrieving module, used to retrieve a preset communication quality reference value; A constraint condition acquisition module, used to set the upper limit value and the lower limit value of the standby communication channel startup constraint condition by using the first constraint condition parameter and the first constraint condition parameter in combination with a preset communication quality reference value; wherein the upper limit value and the lower limit value of the communication channel startup constraint condition are obtained by the following formula: in, S up and S down Respectively represent the upper limit value and the lower limit value of the activation constraint condition of the standby communication channel; S 0 indicates the preset communication quality reference value; Ψ represents a compensation coefficient; and, Ψ Obtained through the following formula: in, C b Indicates the percentage value corresponding to the communication channel saturation.

7. The psychological state recognition system based on facial images and physiological information according to claim 3, characterized in that: The mental state recognition module comprises: An image acquisition control module, used to control the plurality of types of image acquisition devices to acquire real-time facial image data of a user, and to send the facial image data of the user to a data processing platform according to a preset image data sending frequency; A physiological index data acquisition control module, used to use the biosensor group to collect the user's physiological index data in real time, and send the physiological index data to a data processing platform according to a preset physiological index data sending frequency; The psychological state information acquisition module is used to input the target image data and the target physiological indicator data into the trained psychological state recognition model, and identify the user's current psychological state through the trained psychological state recognition model to obtain the user's emotional state data, mental stress level data and mental state data.

8. The psychological state recognition system based on facial images and physiological information according to claim 3, characterized in that: The mental state recognition module comprises: A data acquisition control module, used to control various types of image acquisition devices and biosensor groups to collect image data and physiological index data of the user after the backup communication channel startup constraint conditions are set; A data input module, used for inputting the image data and physiological index data into the trained mental state recognition model; The psychological state recognition module is used to recognize the user's current psychological state through the trained psychological state recognition model, and obtain the user's emotional state data, mental stress level data and mental state data.

9. The psychological state recognition system based on facial images and physiological information according to claim 3, characterized in that: The standby communication channel startup determination module includes: A real-time monitoring module, used for real-time monitoring of data transmission operating parameters during the data transmission process of image data and physiological index data, wherein the data transmission operating parameters include data transmission rate, bit error rate, data delay data and throughput; The communication transmission evaluation parameter acquisition module is used to obtain the communication transmission evaluation parameter using the data transmission operation parameter; wherein the communication transmission evaluation parameter is obtained by the following formula: in, S represents communication transmission evaluation parameters; n Indicates the number of first unit time contained in the current communication duration; b j Indicates j The data transmission rate per first unit time; b e Indicates the standard data transmission rate of a communication channel; p j Indicates j The bit error rate per first unit time; p tj Indicates j The delay rate of the first unit time; C j Indicates j The communication capacity occupancy rate of the first unit time; p sj Indicates j The amount of data with bit errors in the first unit of time delay data accounts for j The proportion of the total amount of data transmission per first unit time; The backup communication channel activation module is used to activate the backup communication channel when the communication transmission evaluation parameter exceeds the constraint range corresponding to the communication channel activation constraint condition.

10. A psychological state recognition method implemented by the psychological state recognition system based on facial images and physiological information as claimed in claim 1, characterized in that: The psychological state recognition method based on facial images and physiological information includes: When there is a user to be identified in the cabin, the image data sending frequency and the physiological index data sending frequency are set according to the user's physiological index data in a preset trial monitoring time period; After the image data transmission frequency and the physiological index data transmission frequency are set, setting the backup communication channel startup constraint conditions according to the data transmission operation parameters of the physiological index data in the trial monitoring time period; After the backup communication channel startup constraint conditions are set, various types of image acquisition devices and biosensor groups are controlled to collect image data and physiological index data of the user and recognize the psychological state; The data transmission operation parameters in the data transmission process of the image data and the physiological index data are monitored in real time, and the data transmission operation parameters and the activation constraint conditions of the backup communication channel are used to determine whether to activate the backup communication channel.

Images