Vehicle-mounted atmosphere lamp control method and device and vehicle

Abstract

The invention discloses a vehicle-mounted atmosphere lamp control method and device and a vehicle, and is applied to the technical field of vehicles, the method comprises the steps that physiological characteristic data of a target person is acquired, the physiological characteristic data comprises a face video, an electromyographic signal sequence, an eyeball image and a pupil image, the electromyographic signal sequence comprises electromyographic signals at a plurality of preset sampling moments, and the electromyographic signals are acquired; the face video comprises a plurality of face images at preset sampling moments; according to the physiological characteristic data, the emotion of the target person is recognized, and target emotion information is obtained; according to the target emotion information, correcting an initial control parameter of the vehicle-mounted atmosphere lamp to obtain a corrected control parameter of the vehicle-mounted atmosphere lamp; and controlling the vehicle-mounted atmosphere lamp according to the corrected control parameters. According to the invention, the adaptive display control of the vehicle-mounted atmosphere lamp can be realized, and the control precision of the vehicle-mounted atmosphere lamp is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device and vehicle for controlling vehicle ambient lighting. Background Art

[0002] The vehicle ambient light is a decorative lighting system used to enhance the interior environment and riding experience of the vehicle. In the related art, the corresponding control mode is usually selected according to the driver's mood, and the vehicle ambient light is controlled according to the selected control mode. However, the control mode of the related art is usually a pre-set control mode, and the control mode of the vehicle ambient light cannot be adaptively adjusted based on the driver's mood, which will cause users to face a series of inconveniences in terms of driving comfort, safety, energy efficiency and personalized experience. Summary of the invention

[0003] The embodiments of the present application provide a vehicle-mounted ambient light control method, device, and vehicle for improving the control accuracy and flexibility of the vehicle-mounted ambient light.

[0004] On the one hand, an embodiment of the present application provides a method for controlling a vehicle ambient light, the method comprising the following steps:

[0005] Acquire physiological characteristic data of the target person, wherein the physiological characteristic data includes a facial video, an electromyographic signal sequence, an eyeball image, and a pupil image, wherein the electromyographic signal sequence includes electromyographic signals at a plurality of preset sampling moments;

[0006] According to the physiological characteristic data, the emotion of the target person is identified to obtain target emotion information;

[0007] According to the target emotion information, the initial control parameters of the vehicle atmosphere light are corrected to obtain the corrected control parameters of the vehicle atmosphere light;

[0008] The vehicle-mounted ambient light is controlled according to the modified control parameter.

[0009] On the other hand, an embodiment of the present application provides a vehicle-mounted ambient light control device, the device comprising:

[0010] An acquisition module, used to acquire physiological characteristic data of a target person, wherein the physiological characteristic data includes a facial video, an electromyographic signal sequence, an eyeball image, and a pupil image, wherein the electromyographic signal sequence includes electromyographic signals at a plurality of preset sampling moments;

[0011] A first processing module is used to identify the emotion of the target person according to the physiological characteristic data to obtain target emotion information;

[0012] A second processing module, configured to correct the initial control parameters of the vehicle-mounted atmosphere light according to the target emotion information, so as to obtain the corrected control parameters of the vehicle-mounted atmosphere light;

[0013] The third processing module is used to control the vehicle ambient light according to the modified control parameter.

[0014] In another aspect, an embodiment of the present application provides a vehicle, the vehicle comprising:

[0015] at least one processor;

[0016] at least one memory for storing at least one program;

[0017] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned vehicle ambient light control method.

[0018] The beneficial effects of the present application are: providing a vehicle-mounted atmosphere light control method, device and vehicle, first, obtaining the physiological characteristic data of the target person, the physiological characteristic data may include facial video, electromyographic signal sequence, eyeball image and pupil image, the electromyographic signal sequence includes electromyographic signals at several preset sampling moments; then, according to the physiological characteristic data, the target person's emotions are identified to obtain the target emotional information; thereafter, according to the target emotional information, the initial control parameters of the vehicle-mounted atmosphere light are corrected to obtain the corrected control parameters of the vehicle-mounted atmosphere light; finally, according to the corrected control parameters, the vehicle-mounted atmosphere light is controlled. The present application characterizes the emotional changes of the target person through multimodal data such as facial video, electromyographic signal sequence, eyeball image and pupil image, and corrects the initial control parameters of the vehicle-mounted atmosphere light when the emotional changes of the target person are fully perceived, and uses the corrected control parameters to realize adaptive display control of the vehicle-mounted atmosphere light, which can improve the control accuracy of the vehicle-mounted atmosphere light and optimize the driving comfort of the vehicle and the driving experience of the user.

[0019] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the vehicle ambient light control method provided by the present application;

[0021] Figure 2 is a schematic diagram of the emotion recognition provided by this application;

[0022] Figure 3It is a schematic diagram of generating a red channel image provided by this application;

[0023] Figure 4 It is a schematic diagram of generating a blue channel image provided by this application;

[0024] Figure 5 It is a schematic diagram of generating a green channel image provided by this application;

[0025] Figure 6 is a structural diagram of the vehicle-mounted ambient light control device provided by the present application;

[0026] Figure 7 is an example diagram of a vehicle provided in this application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0029] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0031] In response to the problems and defects existing in the related technologies, the present application provides a vehicle-mounted ambient light control method, device and vehicle, aiming to achieve adaptive display control of the vehicle-mounted ambient light, ensure the control accuracy of the vehicle-mounted ambient light, and enhance the user's driving experience.

[0032] First, the implementation steps of the vehicle ambient light control method provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0033] The vehicle-mounted atmosphere light control method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, can also be software running in a terminal or a server, etc. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or distributed system built by several physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN) and big data and artificial intelligence platforms. In addition, the server can also be a node server in a blockchain network, but is not limited to this. Among them, blockchain is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.

[0034] Reference Figure 1 , Figure 1 : is a flow chart of the vehicle ambient light control method provided by the present application, and the vehicle ambient light control method may include the following steps S101-S104:

[0035] S101, acquiring physiological characteristic data of a target person, the physiological characteristic data including facial video, electromyographic signal sequence, eyeball image and pupil image, the electromyographic signal sequence including electromyographic signals at several preset sampling moments;

[0036] S102, identifying the target person's emotions based on the physiological characteristic data to obtain target emotion information;

[0037] S103, correcting the initial control parameters of the vehicle ambient light according to the target emotion information to obtain the corrected control parameters of the vehicle ambient light;

[0038] S104, controlling the vehicle ambient light according to the corrected control parameters.

[0039] In the embodiment of the present application, first, the physiological characteristic data of the target person is obtained. The physiological characteristic data may include facial video, electromyographic signal sequence, eyeball image and pupil image. The electromyographic signal sequence includes electromyographic signals at several preset sampling moments; then, the target person's emotions are identified based on the physiological characteristic data, aiming to obtain the target emotional information; then, the initial control parameters of the vehicle atmosphere light are corrected based on the target emotional information to obtain the corrected control parameters of the vehicle atmosphere light; finally, the vehicle atmosphere light is controlled based on the corrected control parameters. It can be seen that the emotional changes of the target person are characterized by multimodal data such as facial video, electromyographic signal sequence, eyeball image and pupil image. When the emotional changes of the target person are fully perceived, the initial control parameters of the vehicle atmosphere light are corrected, and the corrected control parameters are used to realize the adaptive display control of the vehicle atmosphere light. In this way, the control accuracy of the vehicle atmosphere light can be improved, and the driving comfort of the vehicle and the driving experience of the user can be optimized.

[0040] In the above step S101, physiological characteristic data associated with the emotional changes of the target person are obtained, aiming to capture the real emotional expression of the target person through the physiological characteristic data.

[0041] The above-mentioned physiological characteristic data may include facial video, electromyographic signal sequence, eyeball image and pupil image, but is not limited thereto.

[0042] The above-mentioned electromyographic signal sequence may be a one-dimensional time series, and the above-mentioned electromyographic signal sequence may include electromyographic signals at several preset sampling moments, and each electromyographic signal is characterized by a signal amplitude.

[0043] The above-mentioned preset sampling time can be set according to actual conditions, and the embodiments of the present application do not make any specific limitations on this.

[0044] The above-mentioned acquisition of the physiological characteristic data of the target person may include, within a preset time period, performing image acquisition on the face of the target person through at least one preset image acquisition module to obtain a facial video of the target person; and, at any preset sampling time, performing image acquisition on the eyes of the target person through at least one preset image acquisition module to obtain an eyeball image and a pupil image of the target person; and, within a preset time period, performing electromyographic signal acquisition on the fingers of the target person through at least one electromyographic sensor arranged on the steering wheel to obtain an electromyographic signal sequence of the target person, but is not limited to this.

[0045] The above-mentioned preset time period refers to a time period constructed by a number of preset sampling moments.

[0046] The above-mentioned preset time period, the number of the above-mentioned image acquisition modules and the number of the above-mentioned electromyographic sensors can all be set according to actual conditions, and the embodiments of the present application do not make specific limitations on this.

[0047] In the above step S102, the real emotional expression of the target person is captured based on the physiological characteristic data of the target person to obtain the target emotional information, thereby realizing the emotional recognition of the target person.

[0048] The above target emotion information is used to characterize the emotion type of the target person.

[0049] The above emotion types can be set according to actual conditions, and the embodiments of the present application do not make specific limitations on this.

[0050] For example, the above emotion types may include joy, shock, depression, anger, and panic, but are not limited thereto.

[0051] The above-mentioned identifying the target person's emotions based on physiological characteristic data to obtain target emotion information may include identifying the target person's emotions based on physiological characteristic data in combination with a machine learning method to obtain the target person's emotion type as the target emotion information, but is not limited to this.

[0052] The above-mentioned machine learning method can be set according to actual conditions, and the embodiments of the present application do not make any specific limitations on this.

[0053] For example, the above-mentioned machine learning method may be a support vector machine; or, the above-mentioned machine learning method may be a random forest or logistic regression, but is not limited thereto.

[0054] In the above step S103, the initial control parameters of the vehicle ambient light are corrected by using the target emotion information, thereby obtaining the corrected control parameters of the vehicle ambient light, so as to realize adaptive display control of the vehicle ambient light based on the corrected control parameters in subsequent steps.

[0055] The initial control parameters of the above-mentioned vehicle ambient light may include one or more pre-set parameter values ​​or parameter ranges, which are not specifically limited in the embodiments of the present application.

[0056] For example, the initial control parameters of the above-mentioned vehicle ambient light may be parameter values ​​or parameter ranges corresponding to the specific time period to which the current moment belongs. The specific time period may include the time period before sunrise, the time period from sunrise to sunset, or the time period after sunset, etc., but is not limited thereto.

[0057] The above-mentioned revised control parameters of the vehicle-mounted ambient light refer to revised initial control parameters, which are mainly used for adaptively controlling the vehicle-mounted ambient light.

[0058] The above-mentioned method of correcting the initial control parameters of the vehicle atmosphere light according to the target emotion information to obtain the corrected control parameters of the vehicle atmosphere light may include obtaining the corrected control parameters of the vehicle atmosphere light according to the target emotion information and the initial control parameters of the vehicle atmosphere light in combination with a machine learning method, but is not limited to this.

[0059] In the above step S104, after the correction control parameters of the vehicle-mounted ambient light are obtained, adaptive display control of the vehicle-mounted ambient light is implemented by correcting the control parameters.

[0060] The above-mentioned controlling the vehicle ambient light according to the modified control parameters may include controlling the luminous color, luminous brightness and flashing frequency of the vehicle ambient light according to the modified control parameters, but is not limited thereto.

[0061] The above steps will be further described below.

[0062] In some embodiments, the above-mentioned identifying the emotion of the target person based on the physiological characteristic data to obtain the target emotion information may include:

[0063] Performing physiological signal detection on the facial video to obtain a physiological signal sequence, wherein the physiological signal sequence includes physiological signals at a plurality of preset sampling moments, and each physiological signal is a physiological signal based on remote photoplethysmography;

[0064] Extracting features from the physiological signal sequence to obtain a physiological feature sequence, where the physiological feature sequence includes physiological signal features at a number of preset sampling moments;

[0065] Extract features from the eyeball image and pupil image to obtain iris features;

[0066] Extracting features from the electromyographic signal sequence to obtain an electromyographic feature sequence, wherein the electromyographic feature sequence includes electromyographic signal features at a plurality of preset sampling moments;

[0067] According to the physiological feature sequence, iris feature and myoelectric feature sequence, an emotional feature image is obtained;

[0068] The emotion feature image is used as the input of the pre-trained emotion recognition model, and the emotion of the target person is recognized by the emotion recognition model to obtain the target emotion information.

[0069] In this embodiment, the related technology usually inputs the real-time facial image of the target person into a pre-trained neural network model, and uses the neural network model to identify the target person's emotions. However, this method only considers single-modal data, and its emotion recognition accuracy needs to be improved.

[0070] To this end, first, physiological signal detection is performed on the facial video to obtain a physiological signal sequence, which may include physiological signals at several preset sampling moments, and each physiological signal is a physiological signal based on remote photoplethysmography; then, feature extraction is performed on the physiological signal sequence to obtain a physiological feature sequence, which may include physiological signal features at several preset sampling moments; and, feature extraction is performed on the eyeball image and the pupil image to obtain iris features; and, feature extraction is performed on the electromyography signal sequence to obtain an electromyography feature sequence, which may include electromyography signal features at several preset sampling moments; after obtaining the physiological feature sequence, iris features and electromyography feature sequence, imaging processing is performed based on these extracted data to obtain an emotion feature image, which can accurately map the real emotional expression of the target person; finally, the emotion of the target person is recognized through the emotion feature image, thereby obtaining the target emotion information.

[0071] It can be seen that this embodiment extracts the corresponding physiological signal features through the facial video of the target person, extracts the iris features of the target person through the eyeball image and pupil image of the target person, and extracts the electromyography signal features of the target person through the electromyography signal sequence of the target person, and then realizes imaging processing based on these multimodal emotion expression features while retaining the dependence of the data on time, and identifies the emotions of the target person through the imaging results combined with the neural network model. In this way, it is possible to better explore the emotional details hidden in each modal data, capture more and more accurate real emotional expressions, and give full play to the advantages of the neural network model in image recognition tasks, thereby effectively improving the accuracy of emotion recognition.

[0072] The above physiological signal sequence may be a one-dimensional time series.

[0073] The physiological signal sequence may include physiological signals at several preset sampling moments, and the physiological signal may be a physiological signal based on remote photoplethysmography.

[0074] The above physiological characteristic sequence can be a one-dimensional time series.

[0075] The above physiological characteristic sequence may include physiological signal characteristics at several preset sampling moments.

[0076] The above-mentioned physiological signal detection on facial videos to obtain the physiological signal sequence may include performing physiological signal detection on facial videos through a pre-trained first neural network model to obtain physiological signals at several preset sampling moments, and then constructing the physiological signal sequence through the physiological signals at several preset sampling moments, but is not limited to this.

[0077] The first neural network model may be a neural network model trained by a number of facial video samples and a physiological signal sequence corresponding to each facial video sample.

[0078] The above-mentioned first neural network model can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0079] For example, the first neural network model may be a convolutional neural network model; or, the first neural network model may be a Transformer model, but is not limited thereto.

[0080] The above-mentioned feature extraction of the physiological signal sequence to obtain the physiological feature sequence may include extracting the feature of the physiological signal sequence through a first feature extraction model to obtain physiological signal features at several preset sampling moments, and then constructing the physiological feature sequence through the physiological signal features at several preset sampling moments, but is not limited to this.

[0081] The above-mentioned first feature extraction model can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0082] For example, the first feature extraction model may be a recurrent neural network model; or, the first feature extraction model may be a convolutional neural network model, but is not limited thereto.

[0083] The above-mentioned iris feature may be a feature map used to characterize the iris of the target person's eye.

[0084] The above-mentioned feature extraction of the eyeball image and the pupil image to obtain the iris feature may include extracting the feature of the eyeball image to obtain the first iris feature, and extracting the feature of the pupil image to obtain the second iris feature, and then fusing the first iris feature and the second iris feature to obtain the iris feature, but is not limited to this.

[0085] The above electromyographic characteristic sequence can be a one-dimensional time series.

[0086] The above-mentioned electromyographic feature sequence may include electromyographic signal features at several preset sampling moments.

[0087] The above-mentioned electromyographic signal characteristics can be used to characterize the degree of muscle change of the target person, and the degree of muscle change can reflect the emotional changes of the target person.

[0088] The above-mentioned feature extraction of the electromyographic signal sequence to obtain the electromyographic feature sequence may include extracting features of the electromyographic signal sequence through a second feature extraction model to obtain electromyographic signal features at several preset sampling moments, and then constructing the electromyographic signal sequence through the electromyographic signal features at several preset sampling moments, but is not limited to this.

[0089] The second feature extraction model mentioned above can be set according to actual conditions, and this embodiment does not impose any specific limitation on this.

[0090] For example, the second feature extraction model may be a recurrent neural network model; or, the second feature extraction model may be a convolutional neural network model, but is not limited thereto.

[0091] The feature extraction of the physiological signal sequence, the feature extraction of the eyeball image and the pupil image, and the feature extraction of the electromyographic signal sequence may be performed in parallel.

[0092] For example, feature extraction is performed on physiological signal sequences, feature extraction is performed on eyeball images and pupil images, and feature extraction is performed on electromyographic signal sequences.

[0093] Alternatively, the feature extraction of the physiological signal sequence, the feature extraction of the eyeball image and the pupil image, and the feature extraction of the electromyographic signal sequence may be performed in series.

[0094] For example, first, feature extraction is performed on the physiological signal sequence; then, feature extraction is performed on the eyeball image and the pupil image; and then, feature extraction is performed on the electromyographic signal sequence, but it is not limited thereto.

[0095] The above-mentioned emotional feature images are used to map the real emotional expressions of the target persons.

[0096] The above-mentioned method of obtaining the emotion feature image based on the physiological feature sequence, iris feature and electromyographic feature sequence may include performing dimensionality reduction processing on the iris feature to obtain a one-dimensional iris feature, and then fusing the one-dimensional iris feature, electromyographic feature sequence and physiological feature sequence to obtain a fused feature sequence, and then performing dimensionality increase processing on the fused feature sequence to obtain the emotion feature image, but is not limited to this.

[0097] The above-mentioned emotion recognition model can be a neural network model trained by a number of emotion feature image samples and emotion information corresponding to each emotion feature image sample.

[0098] The above emotion recognition model can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0099] For example, the above-mentioned emotion recognition model may be a Faster-RCNN model; or, the above-mentioned emotion recognition model may be a Vision-Transformer model, but is not limited thereto.

[0100] In some embodiments, the facial video may include facial images at a plurality of preset sampling moments; and the physiological signal detection on the facial video to obtain the physiological signal sequence may include:

[0101] Classifying a plurality of facial images to obtain a plurality of first facial images and a plurality of second facial images, wherein the first facial images are facial images having an image quality parameter greater than a preset threshold, and the second facial images are facial images having an image quality parameter less than or equal to the preset threshold;

[0102] Performing physiological signal detection on each first facial image using a pre-trained first physiological signal recognition model to obtain a physiological signal corresponding to each first facial image;

[0103] Performing physiological signal detection on each second facial image through a pre-trained second physiological signal recognition model to obtain a physiological signal corresponding to each second facial image; wherein the accuracy of the first physiological signal recognition model is less than the accuracy of the second physiological signal recognition model, and the architecture parameters of the first physiological signal recognition model are less than the architecture parameters of the second physiological signal recognition model;

[0104] The physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image are spliced ​​to obtain a physiological signal sequence.

[0105] In this embodiment, the act of collecting facial images of the target person usually occurs during the driving process of the vehicle. Affected by driving factors such as road bumps and environmental factors such as ambient lighting, the image quality of some facial images is low, which results in low accuracy of the extracted physiological signals.

[0106] To this end, first, image quality parameters of each facial image are obtained, and the image quality parameters can map the image quality of each facial image, and a number of facial images are classified according to the image quality parameters to obtain multiple first facial images and multiple second facial images, wherein each first facial image is a facial image with higher quality, and each second facial image is a facial image with lower quality; then, physiological signal detection is performed on each first facial image through a pre-trained first physiological signal recognition model to obtain a physiological signal corresponding to each first facial image, and physiological signal detection is performed on each second facial image through a pre-trained second physiological signal recognition model to obtain a physiological signal corresponding to each second facial image, wherein the accuracy of the first physiological signal recognition model is less than the accuracy of the second physiological signal recognition model, and the architecture parameters of the first physiological signal recognition model are less than the architecture parameters of the second physiological signal recognition model; finally, the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image are spliced ​​to obtain a physiological signal sequence. In this way, by using a second physiological signal recognition model with higher accuracy to detect physiological signals of facial images with lower quality, the accuracy of the extracted physiological signals can be ensured, while by using a first physiological signal recognition model with a simpler architecture to detect physiological signals of facial images with higher quality, the detection efficiency of physiological signals can be improved.

[0107] The above image quality parameters can be set according to actual conditions, and this embodiment does not specifically limit this.

[0108] For example, the above-mentioned image quality parameter may be an average structural similarity parameter; or, the above-mentioned image quality parameter may be a peak signal-to-noise ratio parameter, but is not limited thereto.

[0109] The first facial image mentioned above refers to a facial image whose image quality parameter is greater than a preset threshold.

[0110] The second facial image mentioned above refers to a facial image whose image quality parameter is less than or equal to a preset threshold.

[0111] The above preset threshold can be set according to actual conditions, and this embodiment does not specifically limit this.

[0112] Both the first physiological signal recognition model and the second physiological signal recognition model may be neural network models trained by using a number of facial image samples and the physiological signal corresponding to each facial image sample.

[0113] The first physiological signal recognition model and the second physiological signal recognition model can be set according to actual conditions, and this embodiment does not specifically limit this.

[0114] For example, the first physiological signal recognition model mentioned above can be a convolutional neural network model, but is not limited to this.

[0115] For another example, the second physiological signal recognition model may be a Transformer model, but is not limited thereto.

[0116] The physiological signal detection for each first facial image and the physiological signal detection for each second facial image may be performed in parallel.

[0117] For example, physiological signal detection is performed on each first facial image while physiological signal detection is performed on each second facial image.

[0118] Alternatively, the physiological signal detection on each first facial image and the physiological signal detection on each second facial image may be performed in series.

[0119] For example, physiological signal detection is performed on each first facial image first, and then physiological signal detection is performed on each second facial image; or, physiological signal detection is performed on each second facial image first, and then physiological signal detection is performed on each first facial image.

[0120] The above-mentioned splicing processing of the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image to obtain the physiological signal sequence may include randomly splicing the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image into a physiological signal sequence, that is, the position of each physiological signal in the physiological signal sequence is a random position.

[0121] Alternatively, the above-mentioned splicing processing of the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image to obtain the physiological signal sequence may include splicing the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image into a physiological signal sequence according to each preset sampling moment.

[0122] The ranking of the above physiological signals in the physiological signal sequence may be positively correlated with the preset sampling time, that is, the earlier the preset sampling time is, the higher the ranking of the physiological signal in the physiological signal sequence is, and the later the preset sampling time is, the later the ranking of the physiological signal in the physiological signal sequence is.

[0123] Alternatively, the ranking of the above physiological signals in the physiological signal sequence may be negatively correlated with the preset sampling time, that is, the earlier the preset sampling time, the later the ranking of the physiological signal in the physiological signal sequence, and the later the preset sampling time, the earlier the ranking of the physiological signal in the physiological signal sequence.

[0124] In some implementations, the above-mentioned extraction of iris features from the eyeball image and the pupil image to obtain iris features may include:

[0125] Perform iris recognition on the eyeball image to obtain the pupil area in the eyeball image;

[0126] Performing iris recognition on the pupil image to obtain a pupil area in the pupil image;

[0127] According to the pupil area in the eyeball image, the pupil area in the pupil image is corrected to obtain a target pupil area;

[0128] Iris features are extracted from the target pupil area to obtain iris features.

[0129] In this embodiment, first, iris recognition is performed on the eyeball image to locate the pupil area in the eyeball image, and iris recognition is performed on the pupil image to locate the pupil area in the pupil image; then, the pupil area in the eyeball image is used to correct the pupil area in the pupil image, so as to obtain a more accurate pupil area, namely, the target pupil area; finally, iris features are extracted based on the target pupil area, which can effectively improve the extraction accuracy of iris features.

[0130] The above-mentioned performing iris recognition on the eye image to obtain the pupil area in the eye image may include performing iris recognition on the eye image through a pre-trained second neural network model to obtain the pupil area in the eye image, but is not limited to this.

[0131] The second neural network model may be a neural network model trained by a number of eyeball image samples and the pupil area corresponding to each eyeball image sample.

[0132] The above-mentioned second neural network model can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0133] For example, the second neural network model may be a convolutional neural network model; or, the second neural network model may be a Faster-RCNN model, but is not limited thereto.

[0134] Alternatively, the above-mentioned performing iris recognition on the eye image to obtain the pupil area in the eye image may include extracting a plurality of first pupil feature points from the eye image, and then performing image segmentation on the eye image based on the plurality of first pupil feature points in combination with a preset first edge segmentation operator to obtain the pupil area in the eye image.

[0135] The first edge segmentation operator may be set according to actual conditions, and this embodiment does not impose any specific limitation on this.

[0136] For example, the first edge segmentation operator may be a Roberts operator; or, the first edge segmentation operator may be a Sobel operator, but is not limited thereto.

[0137] The above-mentioned performing iris recognition on the pupil image to obtain the pupil area in the pupil image may include performing iris recognition on the pupil image through a pre-trained third neural network model to obtain the pupil area in the pupil image.

[0138] The third neural network model mentioned above may be a neural network model trained by a number of pupil image samples and a pupil area corresponding to each pupil image sample.

[0139] The third neural network model mentioned above can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0140] For example, the third neural network model may be a convolutional neural network model; or, the third neural network model may be a Faster-RCNN model, but is not limited thereto.

[0141] Alternatively, the above-mentioned performing iris recognition on the pupil image to obtain the pupil area in the pupil image may include extracting a plurality of second pupil feature points from the pupil image, and then performing image segmentation on the pupil image based on the plurality of second pupil feature points in combination with a preset second edge segmentation operator to obtain the pupil area in the pupil image.

[0142] The second edge segmentation operator can be set according to actual conditions, and this embodiment does not specifically limit this.

[0143] For example, the second edge segmentation operator may be a Roberts operator; or, the second edge segmentation operator may be a Sobel operator, but is not limited thereto.

[0144] The iris recognition on the eyeball image and the iris recognition on the pupil image may be performed in parallel.

[0145] For example, iris recognition is performed on an eyeball image while iris recognition is performed on a pupil image.

[0146] Alternatively, the iris recognition on the eyeball image and the iris recognition on the pupil image may be performed serially.

[0147] For example, iris recognition is performed on the eyeball image first, and then on the pupil image; or, iris recognition is performed on the pupil image first, and then on the eyeball image.

[0148] The target pupil area refers to the corrected pupil area in the pupil image.

[0149] The above-mentioned correcting the pupil area in the pupil image according to the pupil area in the eyeball image to obtain the target pupil area may include fusing the pupil area in the eyeball image with the pupil area in the pupil image to obtain the target pupil area.

[0150] Alternatively, the above-mentioned correction of the pupil area in the pupil image according to the pupil area in the eyeball image to obtain the target pupil area may include first, projecting the pupil area in the eyeball image onto the pupil area in the pupil image to obtain the projected pupil area as the projected pupil area; then, determining the overlapping area and the non-overlapping area in the projected pupil area, wherein the overlapping area refers to the area where the pupil area in the eyeball image and the pupil area in the pupil image overlap with each other, and each pixel point in the overlapping area is constructed by the pixel points of the pupil area in the eyeball image and the pixel points of the pupil area in the pupil image, and the non-overlapping area refers to the other area except the overlapping area, and each pixel point in the non-overlapping area is the pixel point of the pupil area in the eyeball image or the pixel point of the pupil area in the pupil image; then, for each pixel point in the overlapping area in the projected pupil area, weighted averaging is performed on the pixel points of the pupil area in the eyeball image and the pixel points of the pupil area in the pupil image, while keeping each pixel point in the non-overlapping area in the projected pupil area unchanged, thereby obtaining the processed projected pupil area as the target pupil area, but is not limited to this.

[0151] The above-mentioned extracting iris features from the target pupil area to obtain iris features may include extracting iris features from the target pupil area using a third feature extraction model to obtain iris features, but is not limited thereto.

[0152] The third feature extraction model mentioned above can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0153] For example, the third feature extraction model may be a convolutional neural network model; or, the third feature extraction model may be a deep neural network model, but is not limited thereto.

[0154] In some embodiments, obtaining the emotion feature image according to the physiological feature sequence, iris feature and myoelectric feature sequence may include:

[0155] According to the iris feature and physiological feature sequence, a red channel image is obtained;

[0156] According to the iris feature and electromyography feature sequence, the blue channel image is obtained;

[0157] According to the physiological feature sequence and the electromyographic feature sequence, a green channel image is obtained;

[0158] According to the red channel map, the blue channel map and the green channel map, an emotion feature image is obtained.

[0159] In this embodiment, first, the generation and processing of the red channel image is realized based on the iris feature and the physiological feature sequence, the generation and processing of the blue channel image is realized based on the iris feature and the electromyography feature sequence, and the generation and processing of the green channel image is realized based on the physiological feature sequence and the electromyography feature sequence; then, the image is generated through the red channel image, the blue channel image and the green channel image to obtain the emotion feature image. It can be seen that the three-primary color channel image is obtained through the multimodal feature interaction, and then the three-primary color channel image is used to realize the generation of the emotion feature image, so that the emotional details hidden in each modal data can be better mined, and more and more accurate real emotional expressions can be captured, which is conducive to improving the feature extraction ability of the emotion recognition model.

[0160] The above-mentioned method of obtaining the red channel image based on the iris feature and the physiological feature sequence may include firstly performing dimensionality reduction processing on the iris feature to obtain a one-dimensional iris sequence; then, fusing the one-dimensional iris feature and the physiological feature sequence to obtain a first fused feature; and finally, obtaining the red channel image based on the first fused feature, but is not limited thereto.

[0161] The above-mentioned method of obtaining a blue channel map based on the iris feature and the electromyography feature sequence may include firstly performing dimensionality reduction processing on the iris feature to obtain a one-dimensional iris sequence; then, fusing the one-dimensional iris feature and the electromyography feature sequence to obtain a second fused feature; and finally, obtaining a blue channel map based on the second fused feature, but is not limited thereto.

[0162] The green channel map is obtained according to the physiological feature sequence and the electromyographic feature sequence, which may include fusing the physiological feature sequence and the electromyographic signal sequence to obtain a third fusion feature, and then obtaining the green channel map according to the third fusion feature, but is not limited thereto.

[0163] The acquisition of the red channel image, the acquisition of the blue channel image, and the acquisition of the green channel image may be performed in parallel.

[0164] For example, a red channel image is obtained based on the iris feature and the physiological feature sequence, and a blue channel image is obtained based on the iris feature and the electromyographic feature sequence. At the same time, a green channel image is obtained based on the physiological feature sequence and the electromyographic feature sequence.

[0165] Alternatively, the acquisition of the red channel image, the acquisition of the blue channel image, and the acquisition of the green channel image may be performed serially.

[0166] For example, first, a red channel image is obtained based on iris features and physiological feature sequences; then, a blue channel image is obtained based on iris features and electromyographic feature sequences; and then, a green channel image is obtained based on physiological feature sequences and electromyographic feature sequences, but this is not limited to this.

[0167] The above-mentioned obtaining of the emotion characteristic image according to the red channel map, the blue channel map and the green channel map may include superimposing and reorganizing the red channel map, the blue channel map and the green channel map in the channel dimension to obtain the emotion characteristic image, but is not limited thereto.

[0168] In some embodiments, obtaining a red channel image according to the iris feature and the physiological feature sequence may include:

[0169] Performing pooling processing on the iris features to obtain the first pooling features;

[0170] Performing pooling processing on the physiological feature sequence to obtain a second pooled feature;

[0171] The first pooled feature is processed through the Swin-Transformer module to obtain the first perceptual feature;

[0172] The second pooling feature is processed through the Transformer module to obtain the second perception feature;

[0173] Taking the second perception feature as the query and the iris feature as the key-value pair, the iris feature and the second perception feature are processed through the attention mechanism to obtain the first sub-fusion feature;

[0174] Taking the first perceived feature as the query and the physiological feature sequence as the key-value pair, the physiological feature sequence and the first perceived feature are processed through the attention mechanism to obtain the second sub-fusion feature;

[0175] Fusing the first sub-fusion feature and the second sub-fusion feature to obtain a first fusion feature;

[0176] According to the first fusion feature and in combination with the preset first generation method, a red channel image is obtained.

[0177] In this embodiment, first, the iris features are pooled to obtain the first pooled features to reduce the spatial dimension of the iris features; and the physiological feature sequence is pooled to obtain the second pooled features to compress the channel size of each physiological feature.

[0178] Then, the first pooled features are processed by the Swin-Transformer module to obtain the first perceptual features, which are intended to perceive the visual changes of emotions in the pooled iris features, and the second pooled features are processed by the Transformer module to obtain the second perceptual features, which are intended to perceive the physiological changes of emotions in the pooled physiological feature sequence.

[0179] Afterwards, taking the second perceptual feature as a query and the iris feature as a key-value pair, the iris feature and the second perceptual feature are processed through the attention mechanism, aiming to use the iris feature to adaptively compensate for the feature information of the second perceptual feature at the level of emotional visual changes, and then obtain the first sub-fusion feature; and taking the first perceptual feature as a query and the physiological feature sequence as a key-value pair, the physiological feature sequence and the first perceptual feature are processed through the attention mechanism, aiming to use the physiological feature sequence to adaptively compensate for the feature information of the first perceptual feature at the level of emotional physiological changes, and then obtain the second sub-fusion feature.

[0180] Finally, the first sub-fusion feature and the second sub-fusion feature are fused to obtain a first fusion feature, and a red channel image is obtained based on the first fusion feature in combination with a preset first generation method.

[0181] It can be seen that this embodiment realizes the generation of the red channel map through the multimodal feature interaction of the physiological feature sequence and the iris feature. In this way, it can better mine the detailed information at the level of emotional physiological changes and the level of emotional visual changes, capture more and more accurate real emotional expressions, and help improve the feature extraction capability of the emotion recognition model.

[0182] The above pooling process can be configured according to actual conditions, and this embodiment does not specifically limit this.

[0183] For example, the above-mentioned pooling process for the iris features may include performing maximum pooling process for the iris features to reduce the spatial size of the iris features, but is not limited thereto.

[0184] For another example, the above-mentioned pooling process for the physiological feature sequence may include performing average pooling process on the physiological feature sequence to reduce the channel dimension of the physiological feature sequence, but is not limited thereto.

[0185] The pooling process of the iris features and the pooling process of the physiological feature sequence may be performed in parallel.

[0186] For example, the iris feature is pooled while the physiological feature sequence is pooled.

[0187] Alternatively, the pooling process for the iris features and the pooling process for the physiological feature sequence may be performed in series.

[0188] For example, the iris features are first pooled and then the physiological feature sequence is pooled; or, the physiological feature sequence is first pooled and then the iris features are pooled.

[0189] The above-mentioned Swin-Transformer module refers to a module configured with a Swin-Transformer model.

[0190] The above-mentioned Transformer module refers to a module configured with a Transformer model.

[0191] The processing of the first pooling features and the processing of the second pooling features may be performed in parallel.

[0192] For example, the second pooling feature is processed while the first pooling feature is processed.

[0193] Alternatively, the processing of the first pooling features and the processing of the second pooling features may be performed serially.

[0194] For example, the first pooling feature is processed first, and then the second pooling feature is processed; or, the second pooling feature is processed first, and then the first pooling feature is processed.

[0195] The processing of the iris feature and the second perception feature and the processing of the physiological feature sequence and the first perception feature may be performed in parallel.

[0196] For example, the physiological feature sequence and the first perception feature are processed while the iris feature and the second perception feature are processed.

[0197] Alternatively, the processing of the iris feature and the second perception feature and the processing of the physiological feature sequence and the first perception feature may be performed in series.

[0198] For example, the iris feature and the second perception feature are processed first, and then the physiological feature sequence and the first perception feature are processed; or, the physiological feature sequence and the first perception feature are processed first, and then the iris feature and the second perception feature are processed.

[0199] The fusing process of the first sub-fusion feature and the second sub-fusion feature to obtain the first fusion feature may include adding the first sub-fusion feature and the second sub-fusion feature to obtain the first fusion feature.

[0200] Alternatively, the above-mentioned fusing the first sub-fusion feature and the second sub-fusion feature to obtain the first fusion feature may include superimposing the first sub-fusion feature and the second sub-fusion feature in the channel dimension to obtain the first fusion feature, but is not limited thereto.

[0201] The above-mentioned first generation mode can be set according to actual conditions, and this embodiment does not specifically limit this.

[0202] For example, the first generation method may be a relative displacement matrix diagram, but is not limited thereto.

[0203] In some embodiments, obtaining a blue channel map according to the iris feature and the electromyography feature sequence may include:

[0204] Perform pooling on the iris features to obtain the third pooling feature;

[0205] Performing pooling processing on the electromyographic feature sequence to obtain the fourth pooled feature;

[0206] The third pooling feature is processed by the Swin-Transformer module to obtain the third perceptual feature;

[0207] The fourth pooling feature is processed by the Transformer module to obtain the fourth perceptual feature;

[0208] Taking the fourth perception feature as the query and the iris feature as the key-value pair, the iris feature and the fourth perception feature are processed through the attention mechanism to obtain the third sub-fusion feature;

[0209] The third perception feature is used as a query, and the electromyographic feature sequence is used as a key-value pair. The electromyographic feature sequence and the third perception feature are processed through an attention mechanism to obtain a fourth sub-fusion feature. The third sub-fusion feature and the fourth sub-fusion feature are fused to obtain a second fusion feature.

[0210] According to the second fusion feature and in combination with the preset second generation method, a blue channel map is obtained.

[0211] In this embodiment, first, the iris features are pooled to obtain the third pooled features to reduce the spatial dimension of the iris features; and the electromyographic feature sequence is pooled to obtain the fourth pooled features to compress the channel size of each electromyographic feature.

[0212] Then, the third pooled feature is processed by the Swin-Transformer module to obtain the third perceptual feature, which is intended to perceive the emotional visual changes in the pooled iris features, and the fourth pooled feature is processed by the Transformer module to obtain the fourth perceptual feature, which is intended to perceive the emotional physiological changes in the pooled electromyographic feature sequence.

[0213] Afterwards, taking the fourth perceptual feature as a query and the iris feature as a key-value pair, the iris feature and the fourth perceptual feature are processed through the attention mechanism, aiming to use the iris feature to adaptively compensate for the feature information of the fourth perceptual feature at the level of emotional visual changes, and then obtain the third sub-fusion feature; and taking the third perceptual feature as a query and the electromyography feature sequence as a key-value pair, the electromyography feature sequence and the third perceptual feature are processed through the attention mechanism to obtain the fourth sub-fusion feature, aiming to use the electromyography feature sequence to compensate for the feature information of the third perceptual feature at the level of emotional physiological changes, and then obtain the fourth sub-fusion feature.

[0214] Finally, the third sub-fusion feature and the fourth sub-fusion feature are fused to obtain a second fusion feature, and based on the second fusion feature and combined with a preset second generation method, a blue channel map is obtained.

[0215] It can be seen that this embodiment realizes the generation of the blue channel map through the multimodal feature interaction of the electromyography feature sequence and the iris feature. In this way, it can better mine the detailed information at the level of emotional physiological changes and the level of emotional visual changes, capture more and more accurate real emotional expressions, and help improve the feature extraction capability of the emotion recognition model.

[0216] The above pooling process can be configured according to actual conditions, and this embodiment does not specifically limit this.

[0217] For example, the above-mentioned pooling process for the iris features may include performing maximum pooling process for the iris features to reduce the spatial size of the iris features, but is not limited thereto.

[0218] For another example, the above-mentioned pooling process for the electromyographic feature sequence may include performing average pooling process for the electromyographic feature sequence to reduce the channel dimension of the electromyographic feature sequence, but is not limited thereto.

[0219] The pooling process for the iris features and the pooling process for the electromyography feature sequence may be performed in parallel.

[0220] For example, the iris feature is pooled while the electromyography feature sequence is pooled.

[0221] Alternatively, the pooling process for the iris features and the pooling process for the electromyographic feature sequence may be performed in series.

[0222] For example, the iris features are first pooled and then the electromyography feature sequence is pooled; or, the electromyography feature sequence is first pooled and then the iris features are pooled.

[0223] The processing of the third pooling feature and the processing of the fourth pooling feature may be performed in parallel.

[0224] For example, the fourth pooling feature is processed while the third pooling feature is processed.

[0225] Alternatively, the processing of the third pooling feature and the processing of the fourth pooling feature may be performed serially.

[0226] For example, the third pooling feature is processed first, and then the fourth pooling feature is processed; or, the fourth pooling feature is processed first, and then the third pooling feature is processed.

[0227] The above processing of the iris feature and the fourth perception feature and the above processing of the electromyographic feature sequence and the third perception feature can be performed in parallel.

[0228] For example, the iris feature and the fourth sensory feature are processed while the myoelectric feature sequence and the third sensory feature are processed.

[0229] Alternatively, the above-mentioned processing of the iris feature and the fourth sensory feature and the above-mentioned processing of the electromyographic feature sequence and the third sensory feature can be performed serially.

[0230] For example, the iris feature and the fourth perception feature are processed first, and then the electromyographic feature sequence and the third perception feature are processed; or, the electromyographic feature sequence and the third perception feature are processed first, and then the iris feature and the fourth perception feature are processed.

[0231] The fusing process of the third sub-fusion feature and the fourth sub-fusion feature to obtain the second fusion feature may include adding the third sub-fusion feature and the fourth sub-fusion feature to obtain the second fusion feature.

[0232] Alternatively, the above-mentioned fusing the third sub-fusion feature and the fourth sub-fusion feature to obtain the second fusion feature may include superimposing the third sub-fusion feature and the fourth sub-fusion feature in the channel dimension to obtain the second fusion feature, but is not limited to this.

[0233] The second generation method mentioned above can be set according to actual conditions, and this embodiment does not make any specific limitation to this.

[0234] For example, the second generation method may be a recursive graph, but is not limited thereto.

[0235] In some embodiments, obtaining a green channel image based on the physiological feature sequence and the electromyographic feature sequence may include:

[0236] Performing pooling processing on the physiological feature sequence to obtain the fifth pooled feature;

[0237] Performing pooling processing on the electromyographic feature sequence to obtain the sixth pooled feature;

[0238] The fifth pooling feature is processed through the Transformer module to obtain the fifth perceptual feature;

[0239] The sixth pooling feature is processed through the Transformer module to obtain the sixth perception feature;

[0240] Perform jump connection on the physiological feature sequence and the sixth sensory feature to obtain the fifth sub-fusion feature;

[0241] Perform jump connection on the electromyographic feature sequence and the fifth sensory feature to obtain the sixth sub-fusion feature;

[0242] The fifth sub-fusion feature and the sixth sub-fusion feature are fused to obtain a third fusion feature;

[0243] According to the third fusion feature and in combination with the preset third generation method, a green channel image is obtained.

[0244] In this embodiment, first, the physiological feature sequence and the electromyographic feature sequence are pooled to obtain the fifth pooled feature and the sixth pooled feature to compress the channel dimensions of each physiological feature and each electromyographic feature; then, the fifth pooled feature and the sixth pooled feature are processed by the Transformer module to obtain the fifth perception feature and the sixth perception feature, which are intended to perceive the emotional physiological changes in the physiological feature sequence and the electromyographic feature sequence after pooling; then, the physiological feature sequence and the sixth perception feature are jump-connected to obtain the fifth sub-fusion feature, and the electromyographic feature sequence and the fifth perception feature are jump-connected to obtain the sixth sub-fusion feature, so as to fuse different feature expressions at the level of emotional physiological changes and realize cross-modal feature interaction; finally, the fifth sub-fusion feature and the sixth sub-fusion feature are fused to obtain the third fusion feature, and according to the third fusion feature, combined with the preset third generation method, a green channel map is obtained. It can be seen that the generation of the green channel map is realized by the multimodal feature interaction of the physiological feature sequence and the electromyographic feature sequence, so that the detailed information at the level of emotional physiological changes can be better mined, more and more accurate real emotional expressions can be captured, and the feature extraction ability of the emotion recognition model can be improved.

[0245] The above pooling process can be configured according to actual conditions, and this embodiment does not specifically limit this.

[0246] For example, the above-mentioned pooling process for the physiological feature sequence may include performing average pooling process on the physiological feature sequence to reduce the channel dimension of the physiological feature sequence, but is not limited thereto.

[0247] For another example, the above-mentioned pooling process for the electromyographic feature sequence may include performing average pooling process for the electromyographic feature sequence to reduce the channel dimension of the electromyographic feature sequence, but is not limited thereto.

[0248] The above-mentioned pooling process for the electromyographic feature sequence and the above-mentioned pooling process for the physiological feature sequence may be performed in parallel.

[0249] For example, the electromyographic feature sequence is pooled while the physiological feature sequence is pooled.

[0250] Alternatively, the pooling process for the electromyographic feature sequence and the pooling process for the physiological feature sequence may be performed in series.

[0251] For example, the electromyographic feature sequence is first pooled, and then the physiological feature sequence is pooled; or, the physiological feature sequence is first pooled, and then the electromyographic feature sequence is pooled.

[0252] The processing of the fifth pooling feature and the processing of the sixth pooling feature may be performed in parallel.

[0253] For example, the sixth pooling feature is processed while the fifth pooling feature is processed.

[0254] Alternatively, the processing of the fifth pooling feature and the processing of the sixth pooling feature may be performed serially.

[0255] For example, the fifth pooling feature is processed first, and then the sixth pooling feature is processed; or, the sixth pooling feature is processed first, and then the fifth pooling feature is processed.

[0256] The above-mentioned skip connection between the physiological feature sequence and the sixth sensory feature and the above-mentioned skip connection between the electromyographic feature sequence and the fifth sensory feature can be performed in parallel.

[0257] For example, a jump connection is made between the physiological feature sequence and the sixth sensory feature, while a jump connection is made between the electromyographic feature sequence and the fifth sensory feature.

[0258] Alternatively, the above-mentioned skip connection between the physiological feature sequence and the sixth sensory feature and the above-mentioned skip connection between the electromyographic feature sequence and the fifth sensory feature can be performed in series.

[0259] For example, the physiological feature sequence and the sixth perception feature are first jump-connected, and then the electromyographic feature sequence and the fifth perception feature are jump-connected; or, the electromyographic feature sequence and the fifth perception feature are first jump-connected, and then the physiological feature sequence and the sixth perception feature are jump-connected.

[0260] The fusing process of the fifth sub-fusion feature and the sixth sub-fusion feature to obtain the third fusion feature may include adding the fifth sub-fusion feature and the sixth sub-fusion feature to obtain the third fusion feature.

[0261] Alternatively, the above-mentioned fusing the fifth sub-fusion feature and the sixth sub-fusion feature to obtain the third fusion feature may include superimposing the fifth sub-fusion feature and the sixth sub-fusion feature in the channel dimension to obtain the third fusion feature, but is not limited to this.

[0262] The third generation method mentioned above can be set according to actual conditions, and this embodiment does not specifically limit this.

[0263] For example, the third generation method may be a Gram and angle field matrix, but is not limited thereto.

[0264] In some implementations, the above-mentioned correction of the initial control parameters of the vehicle atmosphere light according to the target emotion information to obtain the corrected control parameters of the vehicle atmosphere light may include:

[0265] According to the target emotion information, the correction coefficient of the vehicle ambient light is obtained;

[0266] According to the correction coefficient, the initial control parameters of the vehicle ambient light are corrected to obtain the corrected control parameters of the vehicle ambient light.

[0267] In this embodiment, after obtaining the target emotion information, first, the correction coefficient of the vehicle atmosphere light is determined based on the target emotion information; then, according to the correction coefficient, the initial control parameters of the vehicle atmosphere light are corrected to obtain the corrected control parameters of the vehicle atmosphere light, thereby improving the control accuracy of the vehicle atmosphere light through correction processing.

[0268] The correction coefficient of the vehicle ambient light is used to correct the initial control parameters of the vehicle ambient light.

[0269] The above-mentioned obtaining of the correction coefficient of the vehicle atmosphere light according to the target emotion information may include searching for the correction coefficient corresponding to the target emotion information from the first correction mapping data, wherein the first correction mapping data may include several preset emotion information and the correction coefficient corresponding to each preset emotion information, but is not limited to this.

[0270] The first corrected mapping data may be chart data or table data, but is not limited thereto.

[0271] The above-mentioned correcting the initial control parameters of the vehicle atmosphere light according to the correction coefficient to obtain the corrected control parameters of the vehicle atmosphere light may include calculating the sum of the correction coefficient and the initial control parameters of the vehicle atmosphere light as the corrected control parameters of the vehicle atmosphere light, but is not limited to this.

[0272] In some embodiments, the correction coefficient may include at least one of a color temperature correction coefficient, a brightness correction coefficient, or a flicker frequency correction coefficient; the correction coefficient of the vehicle atmosphere light obtained according to the target emotion information may include at least one of the following:

[0273] According to the target emotion information and the vehicle's interior temperature parameters, a color temperature correction coefficient is obtained;

[0274] A brightness correction coefficient is obtained according to target emotion information, operating parameters of the vehicle ambient light, and light intensity parameters outside the vehicle; the operating parameters of the vehicle ambient light include input power and output power of the vehicle ambient light;

[0275] According to the target emotion information and operating parameters, the flicker frequency correction coefficient is obtained.

[0276] In this embodiment, the correction coefficient for correcting the initial control parameter is determined from at least one of the color temperature, brightness and flickering frequency of the vehicle-mounted ambient light. Specifically, the color temperature correction coefficient is determined using the target emotion information and the vehicle's in-vehicle temperature parameter, and the color temperature correction coefficient is used to correct the initial control parameter associated with the color temperature of the vehicle-mounted ambient light; and the brightness correction coefficient is determined using the target emotion information, the operating parameters of the vehicle-mounted ambient light and the vehicle's external light intensity parameter, and the brightness correction coefficient is used to correct the initial control parameter associated with the brightness of the vehicle-mounted ambient light; and the flickering frequency correction coefficient is determined using the target emotion information and the operating parameters, and the flickering frequency correction coefficient is used to correct the initial control parameter associated with the flickering frequency of the vehicle-mounted ambient light. In this way, the vehicle's in-car temperature parameters and outside light intensity parameters can reflect the vehicle's internal and external environmental conditions, and the operating parameters of the vehicle's ambient light can reflect the power conditions of the vehicle's ambient light. While fully considering the vehicle's internal and external environmental conditions and the power conditions of the vehicle's ambient light, at least one of the color temperature correction coefficient, brightness correction coefficient or flicker frequency correction coefficient is determined to facilitate subsequent parameter correction processing, thereby improving the correction accuracy of the initial control parameters of the vehicle's ambient light.

[0277] The above-mentioned vehicle interior temperature parameters can be detected and obtained by a preset temperature sensor, but are not limited to this.

[0278] The color temperature correction coefficient is mainly used to correct the initial control parameters associated with the color temperature of the vehicle ambient light.

[0279] The color temperature correction coefficient is obtained according to the target emotion information and the vehicle interior temperature parameter, and may include obtaining the color temperature correction coefficient according to the target emotion information and the vehicle interior temperature parameter in combination with a machine learning method.

[0280] Alternatively, obtaining the color temperature correction coefficient based on the target emotion information and the vehicle's interior temperature parameters may include searching for a correction coefficient associated with the target emotion information and the vehicle's interior temperature parameters as the color temperature correction coefficient from second correction mapping data, wherein the second correction mapping data may include multiple preset first variables and a correction coefficient corresponding to each preset first variable, and each preset first variable may include preset emotion information and a preset temperature parameter, but is not limited to this.

[0281] The second corrected mapping data may be chart data or table data, but is not limited thereto.

[0282] The above-mentioned vehicle exterior light intensity parameters can be detected and obtained by a preset light sensor, but are not limited thereto.

[0283] The brightness correction coefficient is mainly used to correct the initial control parameters associated with the brightness of the vehicle ambient light.

[0284] The operating parameters of the above-mentioned vehicle ambient light may include the input power and output power of the vehicle ambient light, but are not limited thereto.

[0285] The above-mentioned obtaining of the brightness correction coefficient based on the target emotion information, the operating parameters of the vehicle ambient light and the vehicle's outdoor light intensity parameters may include obtaining the brightness correction coefficient based on the target emotion information, the operating parameters of the vehicle ambient light and the vehicle's outdoor light intensity parameters in combination with a machine learning method.

[0286] Alternatively, the brightness correction coefficient obtained according to the target emotional information, the operating parameters of the vehicle-mounted ambient light and the vehicle's outdoor light intensity parameters may include searching for a correction coefficient corresponding to the target emotional information, the operating parameters of the vehicle-mounted ambient light and the vehicle's outdoor light intensity parameters from third correction mapping data as the brightness correction coefficient, wherein the third correction mapping data may include multiple preset second variables and a correction coefficient corresponding to each preset second variable, each preset second variable including preset emotional information, preset operating parameters and preset light intensity parameters, but is not limited to this.

[0287] The third modified mapping data may be chart data or table data, but is not limited thereto.

[0288] The above flicker frequency correction coefficient is mainly used to correct the initial control parameters associated with the flicker frequency of the vehicle ambient light.

[0289] The obtaining of the flicker frequency correction coefficient according to the target emotion information and the operating parameters may include obtaining the flicker frequency correction coefficient according to the target emotion information and the operating parameters in combination with a machine learning method.

[0290] Alternatively, obtaining the flicker frequency correction coefficient based on the target emotional information and operating parameters may include searching for a correction coefficient corresponding to the target emotional information and operating parameters from fourth correction mapping data as the flicker frequency correction coefficient, wherein the fourth correction mapping data may include multiple preset third variables and a correction coefficient corresponding to each preset third variable, and each preset third variable may include preset emotional information and preset operating parameters, but is not limited to this.

[0291] The fourth modified mapping data may be chart data or table data, but is not limited thereto.

[0292] In some embodiments, the initial control parameter may include at least one of an initial brightness range, an initial color temperature range, or an initial flicker frequency range, and the modified control parameter may include at least one of a modified brightness range, a modified color temperature range, or a modified flicker frequency range;

[0293] The initial control parameters of the vehicle ambient light are corrected according to the correction coefficient to obtain the corrected control parameters of the vehicle ambient light, which may include at least one of the following:

[0294] Obtaining the product of the color temperature correction coefficient and the initial color temperature range as the corrected color temperature range;

[0295] Obtaining the product of the brightness correction coefficient and the initial brightness range as the corrected brightness range;

[0296] The product of the flicker frequency correction coefficient and the initial flicker frequency range is obtained as the corrected flicker frequency range.

[0297] In this embodiment, the initial control parameters of the vehicle-mounted atmosphere light are corrected from at least one of the color temperature, brightness and flickering frequency of the vehicle-mounted atmosphere light. Specifically, the product of the color temperature correction coefficient and the initial color temperature range is obtained as the corrected color temperature range, and the corrected color temperature range is used to control the color temperature of the vehicle-mounted atmosphere light; and the product of the brightness correction coefficient and the initial brightness range is obtained as the corrected brightness range, and the corrected brightness range is used to control the brightness of the vehicle-mounted atmosphere light; and the product of the flickering frequency correction coefficient and the initial flickering frequency range is obtained as the corrected flickering frequency range, and the corrected flickering frequency range is used to control the flickering frequency of the vehicle-mounted atmosphere light, so that the control accuracy of the vehicle-mounted atmosphere light is improved through correction processing.

[0298] The above-mentioned initial color temperature range refers to the initial control parameters associated with the color temperature of the vehicle ambient light.

[0299] The above-mentioned corrected color temperature range refers to the corrected initial color temperature range, which is used to control the color temperature of the vehicle-mounted ambient light.

[0300] The above-mentioned initial brightness range refers to the initial control parameter associated with the brightness of the vehicle ambient light.

[0301] The above-mentioned corrected brightness range refers to the corrected initial brightness range, which is used to control the brightness of the vehicle ambient light.

[0302] The above-mentioned initial flickering frequency range refers to the initial control parameters associated with the flickering frequency of the vehicle ambient light.

[0303] The above-mentioned corrected flicker frequency range refers to the corrected initial flicker frequency range, which is used to control the flicker frequency of the vehicle-mounted ambient light.

[0304] In some implementations, the controlling of the vehicle ambient light according to the modified control parameter may include at least one of the following:

[0305] Adjust the color temperature of the vehicle ambient light according to the corrected color temperature range;

[0306] Adjust the brightness of the vehicle ambient light according to the corrected brightness range;

[0307] Adjust the flashing frequency of the vehicle ambient light according to the corrected flashing frequency range.

[0308] In this embodiment, the vehicle-mounted atmosphere light is controlled from at least one of its color temperature, brightness, and flickering frequency. Specifically, the color temperature of the vehicle-mounted atmosphere light is adjusted according to the corrected color temperature range so that the real-time color temperature of the vehicle-mounted atmosphere light is within the corrected color temperature range; and, according to the corrected brightness range, the brightness of the vehicle-mounted atmosphere light is adjusted so that the real-time brightness of the vehicle-mounted atmosphere light is within the corrected brightness range; and, according to the corrected flickering frequency range, the flickering frequency of the vehicle-mounted atmosphere light is adjusted so that the real-time flickering frequency of the vehicle-mounted atmosphere light is within the corrected flickering frequency range, thereby realizing adaptive display control of the vehicle-mounted atmosphere light.

[0309] In order to facilitate understanding of the vehicle ambient light control method of the present application, the vehicle ambient light control method of the present application will be described below by way of example using a practical application scenario. The vehicle ambient light control process of this example specifically includes the following steps S201-S205.

[0310] S201, within a preset time period, at least one preset image acquisition module is used to acquire an image of the target person's face to obtain a facial video of the target person; at the same time, at any preset sampling time, at least one preset image acquisition module is used to acquire an image of the target person's eyes to obtain an eyeball image and a pupil image; at the same time, within a preset time period, at least one electromyographic sensor disposed on the steering wheel is used to acquire electromyographic signals of the target person's fingers to obtain an electromyographic signal sequence of the target person. The target person is a driver.

[0311] S202, identifying the target person's emotions based on the physiological characteristic data to obtain target emotion information. Figure 2 , this step may include the following steps S2021-S2025.

[0312] S2021, perform physiological signal detection on the facial video to obtain a physiological signal sequence, and perform feature extraction on the physiological signal sequence through a convolutional neural network model to obtain a physiological feature sequence, where the physiological feature sequence includes physiological signal features at several preset sampling moments.

[0313] For the detection of physiological signals, firstly, the image quality parameters of each facial image are obtained, and the image quality parameters can map the image quality of each facial image. According to the image quality parameters, a plurality of facial images are classified to obtain a plurality of first facial images and a plurality of second facial images, wherein each first facial image is a facial image of higher quality, and each second facial image is a facial image of lower quality. Then, the physiological signal detection is performed on each first facial image through a pre-trained first physiological signal recognition model to obtain the physiological signal corresponding to each first facial image. Meanwhile, the physiological signal detection is performed on each second facial image through a pre-trained second physiological signal recognition model to obtain the physiological signal corresponding to each second facial image, wherein the accuracy of the first physiological signal recognition model is less than the accuracy of the second physiological signal recognition model, and the architecture parameters of the first physiological signal recognition model are less than the architecture parameters of the second physiological signal recognition model. Finally, according to the sequence of preset sampling moments, the physiological signals corresponding to each first facial image and the physiological signals corresponding to each second facial image are spliced ​​to obtain a physiological signal sequence.

[0314] S2022, first, perform iris recognition on the eyeball image through a pre-trained second neural network model to obtain the pupil area in the eyeball image, and at the same time, perform iris recognition on the pupil image through a pre-trained third neural network model to obtain the pupil area in the pupil image; then, based on the pupil area in the eyeball image, correct the pupil area in the pupil image to obtain the target pupil area; finally, perform iris feature extraction on the target pupil area through a convolutional neural network model to obtain iris features.

[0315] For correction of the pupil area, first, the pupil area in the eyeball image is projected onto the pupil area in the pupil image, and the projected pupil area is obtained as the projected pupil area; then, the overlapping area and the non-overlapping area in the projected pupil area are determined, wherein the overlapping area refers to the area where the pupil area in the eyeball image and the pupil area in the pupil image overlap with each other, and each pixel point in the overlapping area is constructed by the pixel points of the pupil area in the eyeball image and the pixel points of the pupil area in the pupil image, and the non-overlapping area refers to the area other than the overlapping area, and each pixel point in the non-overlapping area is the pixel point of the pupil area in the eyeball image or the pixel point of the pupil area in the pupil image; then, for each pixel point in the overlapping area in the projected pupil area, the pixel points of the pupil area in the eyeball image and the pixel points of the pupil area in the pupil image are weighted averaged, while the pixel points of the non-overlapping area in the projected pupil area are kept unchanged, thereby obtaining the processed projected pupil area as the target pupil area.

[0316] S2023, extracting features of the electromyographic signal sequence through a convolutional neural network model to obtain an electromyographic feature sequence, where the electromyographic feature sequence includes electromyographic signal features at several preset sampling moments.

[0317] S2024, obtain a red channel image based on the iris feature and the physiological feature sequence; at the same time, obtain a blue channel image based on the iris feature and the electromyography feature sequence; at the same time, obtain a green channel image based on the physiological feature sequence and the electromyography feature sequence; then, superimpose and reorganize the red channel image, the blue channel image and the green channel image in the channel dimension to obtain an emotion feature image.

[0318] For the generation of the red channel image, refer to Figure 3 First, the iris feature is pooled to obtain the first pooled feature, and the physiological feature sequence is pooled to obtain the second pooled feature; then, the first pooled feature is processed by the Swin-Transformer module to obtain the first perceptual feature, and the second pooled feature is processed by the Transformer module to obtain the second perceptual feature; then, the second perceptual feature is used as a query and the iris feature is used as a key-value pair, and the iris feature and the second perceptual feature are processed by the attention mechanism to obtain the first sub-fusion feature, and the first perceptual feature is used as a query and the physiological feature sequence is used as a key-value pair, and the physiological feature sequence and the first perceptual feature are processed by the attention mechanism to obtain the second sub-fusion feature; finally, the first sub-fusion feature and the second sub-fusion feature are added to obtain the first fusion feature, and the red channel map is obtained according to the first fusion feature and combined with the relative displacement matrix map.

[0319] For the generation of the blue channel image, refer to Figure 4 First, the iris feature is pooled to obtain the third pooled feature, and the electromyographic feature sequence is pooled to obtain the fourth pooled feature; then, the third pooled feature is processed by the Swin-Transformer module to obtain the third perceptual feature, and the fourth pooled feature is processed by the Transformer module to obtain the fourth perceptual feature; then, the fourth perceptual feature is used as a query and the iris feature is used as a key-value pair, and the iris feature and the fourth perceptual feature are processed by the attention mechanism to obtain the third sub-fusion feature, and the third perceptual feature is used as a query and the electromyographic feature sequence is used as a key-value pair, and the electromyographic feature sequence and the third perceptual feature are processed by the attention mechanism to obtain the fourth sub-fusion feature; finally, the third sub-fusion feature and the fourth sub-fusion feature are added to obtain the second fusion feature, and according to the second fusion feature, the recursive graph is combined to obtain the blue channel graph.

[0320] For the generation of the green channel image, refer to Figure 5First, the physiological feature sequence is pooled to obtain the fifth pooled feature, and the electromyographic feature sequence is pooled to obtain the sixth pooled feature; then, the fifth pooled feature is processed by the Transformer module to obtain the fifth perceptual feature, and the sixth pooled feature is processed by the Transformer module to obtain the sixth perceptual feature; then, the physiological feature sequence and the sixth perceptual feature are jump-connected to obtain the fifth sub-fusion feature, and the electromyographic feature sequence and the fifth perceptual feature are jump-connected to obtain the sixth sub-fusion feature; finally, the fifth sub-fusion feature and the sixth sub-fusion feature are added to obtain the third fusion feature, and according to the third fusion feature, combined with the Gram and angular field matrices, the green channel map is obtained.

[0321] S2025, using the emotion feature image as the input of a pre-trained emotion recognition model, identifying the emotion of the target person through the emotion recognition model, and obtaining target emotion information.

[0322] S203, first, obtain the vehicle's in-car temperature parameters through a temperature sensor, obtain the vehicle's outdoor light intensity parameters through a light sensor, and obtain the operating parameters of the vehicle's ambient light; then, based on the target emotional information and the vehicle's in-car temperature parameters, combined with a machine learning method, obtain a color temperature correction coefficient; and / or, based on the target emotional information, the operating parameters of the vehicle's ambient light and the vehicle's outdoor light intensity parameters, combined with a machine learning method, obtain a brightness correction coefficient; and / or, based on the target emotional information and the operating parameters, combined with a machine learning method, obtain a flicker frequency correction coefficient.

[0323] S204, obtaining the product of the color temperature correction coefficient and the initial color temperature range as the corrected color temperature range; and / or, obtaining the product of the brightness correction coefficient and the initial brightness range as the corrected brightness range; and / or, obtaining the product of the flicker frequency correction coefficient and the initial flicker frequency range as the corrected flicker frequency range.

[0324] S205, adjust the color temperature of the vehicle atmosphere light according to the corrected color temperature range, so that the real-time color temperature of the vehicle atmosphere light is within the corrected color temperature range; and / or, adjust the brightness of the vehicle atmosphere light according to the corrected brightness range, so that the real-time brightness of the vehicle atmosphere light is within the corrected brightness range; and / or, adjust the flickering frequency of the vehicle atmosphere light according to the corrected flickering frequency range, so that the real-time flickering frequency of the vehicle atmosphere light is within the corrected flickering frequency range.

[0325] Secondly, the implementation method of the vehicle atmosphere light control device provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0326] Reference Figure 6 The vehicle-mounted ambient light control device provided in the embodiment of the present application mainly includes:

[0327] An acquisition module 301 is used to acquire physiological characteristic data of a target person, wherein the physiological characteristic data includes a facial video, an electromyographic signal sequence, an eyeball image, and a pupil image, wherein the electromyographic signal sequence includes electromyographic signals at a plurality of preset sampling moments, and the facial video includes facial images at a plurality of preset sampling moments;

[0328] The first processing module 302 is used to identify the emotion of the target person according to the physiological characteristic data to obtain the target emotion information;

[0329] The second processing module 303 is used to correct the initial control parameters of the vehicle atmosphere light according to the target emotion information to obtain the corrected control parameters of the vehicle atmosphere light;

[0330] The third processing module 304 is used to control the vehicle ambient light according to the modified control parameter.

[0331] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0332] In addition, refer to Figure 7 , the embodiment of the present application further provides a vehicle, which may include:

[0333] at least one processor 401;

[0334] At least one memory 402, used to store at least one program;

[0335] When at least one program is executed by at least one processor 401, at least one processor 401 implements the above-mentioned vehicle ambient light control method.

[0336] The above-mentioned vehicles can be private cars, such as sedans, sport utility vehicles (SUV), multi-purpose vehicles (MPV) or pickup trucks, etc., or they can be commercial vehicles, such as vans, buses, small trucks or large trailers, etc. They can also be gasoline vehicles or new energy vehicles such as hybrid and pure electric vehicles.

[0337] The above-mentioned memory 402, as a non-transient network system, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory 402 remotely arranged relative to the processor 401, and these remote memories 402 may be connected to the processor 401 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0338] The above-mentioned memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 402 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 402, and the processor 401 is called to execute the method of the embodiment of this application.

[0339] The processor 401 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in the embodiments of the present application.

[0340] In some embodiments, the vehicle may further include:

[0341] Input / output interface, used to realize information input and output;

[0342] Communication interface, used to realize communication interaction between this device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0343] A bus that transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface, and communication interface);

[0344] The processor 401 , the memory 402 , the input / output interface and the communication interface may be communicatively connected to each other within the device via a bus.

[0345] The contents of the above method embodiments are all applicable to the present vehicle embodiment. The functions specifically implemented by the present vehicle embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0346] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0347] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Technical personnel familiar with the field may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for controlling a vehicle ambient light, characterized in that: The following steps are involved: Acquire physiological characteristic data of the target person, wherein the physiological characteristic data includes a facial video, an electromyographic signal sequence, an eyeball image, and a pupil image, wherein the electromyographic signal sequence includes electromyographic signals at a plurality of preset sampling moments; According to the physiological characteristic data, the emotion of the target person is identified to obtain target emotion information; According to the target emotion information, the initial control parameters of the vehicle atmosphere light are corrected to obtain the corrected control parameters of the vehicle atmosphere light; The vehicle-mounted ambient light is controlled according to the modified control parameter.

2. The vehicle-mounted ambient light control method according to claim 1, characterized in that: The step of identifying the target person's emotions based on the physiological characteristic data to obtain target emotion information includes: Performing physiological signal detection on the facial video to obtain a physiological signal sequence, wherein the physiological signal sequence includes physiological signals at a plurality of preset sampling moments, and each of the physiological signals is a physiological signal based on remote photoplethysmography; Extracting features from the physiological signal sequence to obtain a physiological feature sequence, wherein the physiological feature sequence includes physiological signal features at a plurality of preset sampling moments; Extracting features from the eyeball image and the pupil image to obtain iris features; Extracting features from the electromyographic signal sequence to obtain an electromyographic feature sequence, wherein the electromyographic feature sequence includes electromyographic signal features at a plurality of preset sampling moments; Obtaining an emotion feature image according to the physiological feature sequence, the iris feature and the myoelectric feature sequence; The emotion feature image is used as the input of a pre-trained emotion recognition model, and the emotion of the target person is recognized through the emotion recognition model to obtain the target emotion information.

3. The vehicle-mounted ambient light control method according to claim 2, characterized in that: The facial video includes a plurality of facial images at the preset sampling moments; the physiological signal detection is performed on the facial video to obtain a physiological signal sequence, including: Classifying the plurality of facial images to obtain a plurality of first facial images and a plurality of second facial images, wherein the first facial images are facial images having an image quality parameter greater than a preset threshold, and the second facial images are facial images having an image quality parameter less than or equal to the preset threshold; Performing physiological signal detection on each of the first facial images using a pre-trained first physiological signal recognition model to obtain a physiological signal corresponding to each of the first facial images; Performing physiological signal detection on each of the second facial images through a pre-trained second physiological signal recognition model to obtain a physiological signal corresponding to each of the second facial images; wherein the accuracy of the first physiological signal recognition model is less than the accuracy of the second physiological signal recognition model, and the architecture parameters of the first physiological signal recognition model are less than the architecture parameters of the second physiological signal recognition model; The physiological signals corresponding to each of the first facial images and the physiological signals corresponding to each of the second facial images are spliced ​​to obtain the physiological signal sequence.

4. The vehicle-mounted ambient light control method according to claim 2, characterized in that: The extracting iris features from the eyeball image and the pupil image to obtain iris features includes: Performing iris recognition on the eyeball image to obtain a pupil area in the eyeball image; Performing iris recognition on the pupil image to obtain a pupil area in the pupil image; According to the pupil area in the eyeball image, the pupil area in the pupil image is corrected to obtain a target pupil area; Iris features are extracted from the target pupil area to obtain the iris features.

5. The vehicle-mounted ambient light control method according to claim 2, characterized in that: The step of obtaining an emotion feature image according to the physiological feature sequence, the iris feature and the electromyographic feature sequence comprises: Obtaining a red channel image according to the iris feature and the physiological feature sequence; Obtaining a blue channel image according to the iris feature and the electromyography feature sequence; Obtaining a green channel image according to the physiological feature sequence and the electromyographic feature sequence; The emotion characteristic image is obtained according to the red channel image, the blue channel image and the green channel image.

6. The vehicle-mounted ambient light control method according to claim 1, characterized in that: The step of correcting the initial control parameters of the vehicle-mounted atmosphere light according to the target emotion information to obtain the corrected control parameters of the vehicle-mounted atmosphere light includes: Obtaining a correction coefficient of the vehicle-mounted atmosphere light according to the target emotion information; The initial control parameters of the vehicle-mounted atmosphere light are corrected according to the correction coefficient to obtain the corrected control parameters of the vehicle-mounted atmosphere light.

7. The vehicle-mounted ambient light control method according to claim 6, characterized in that: The correction coefficient includes at least one of a color temperature correction coefficient, a brightness correction coefficient, or a flicker frequency correction coefficient; the correction coefficient of the vehicle atmosphere light obtained according to the target emotion information includes at least one of the following: Obtaining the color temperature correction coefficient according to the target emotion information and the vehicle interior temperature parameter; Obtaining the brightness correction coefficient according to the target emotion information, the operating parameters of the vehicle-mounted ambient light, and the external light intensity parameters of the vehicle; The operating parameters of the vehicle-mounted ambient light include the input power and output power of the vehicle-mounted ambient light; The flicker frequency correction coefficient is obtained according to the target emotion information and the operating parameters.

8. The vehicle-mounted ambient light control method according to claim 7, characterized in that: The initial control parameter includes at least one of an initial brightness range, an initial color temperature range, or an initial flicker frequency range, and the modified control parameter includes at least one of a modified brightness range, a modified color temperature range, or a modified flicker frequency range; the initial control parameter of the vehicle atmosphere lamp is modified according to the correction coefficient to obtain the modified control parameter of the vehicle atmosphere lamp, including at least one of the following: Obtaining the product of the color temperature correction coefficient and the initial color temperature range as the corrected color temperature range; Obtaining the product of the brightness correction coefficient and the initial brightness range as the corrected brightness range; The product of the flicker frequency correction coefficient and the initial flicker frequency range is obtained as the corrected flicker frequency range.

9. A vehicle-mounted ambient light control device, characterized in that: include: An acquisition module, used to acquire physiological characteristic data of a target person, wherein the physiological characteristic data includes a facial video, an electromyographic signal sequence, an eyeball image, and a pupil image, wherein the electromyographic signal sequence includes electromyographic signals at a plurality of preset sampling moments; A first processing module is used to identify the emotion of the target person according to the physiological characteristic data to obtain target emotion information; A second processing module, configured to correct the initial control parameters of the vehicle-mounted atmosphere light according to the target emotion information, so as to obtain the corrected control parameters of the vehicle-mounted atmosphere light; The third processing module is used to control the vehicle ambient light according to the modified control parameter.

10. A vehicle, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the vehicle ambient light control method as described in any one of claims 1-8.

Images