Emotion recognition method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a kind of mood recognition method, electronic equipment and storage medium, belong to artificial intelligence technical field.The method comprises: according to the processor frequency, the power level of the pre-set electronic equipment is evaluated.Original mood recognition model is divided into intermediate model section.According to the power level, the target sparsity of each intermediate model section is calculated, and the intermediate model section is compressed to obtain compressed model section according to the target sparsity.According to the processor processing bit width, the weight of compressed network is grouped to obtain weight grouping, and the weight grouping is packaged as target model section.Determine current model section from each target model section, load current model section to the shared memory of pre-set electronic equipment, carry out mood recognition based on pre-set face image by current model section, and eliminate the model section of last loading from shared memory.This application can be compatible with electronic equipment of different hardware configurations, to realize the efficient local operation of mood recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an emotion recognition method, electronic device, and storage medium. Background Technology

[0002] To meet users' growing demand for intelligent features on electronic devices, most mobile applications implement emotion recognition based on facial images using artificial intelligence models. This allows for various intelligent functions based on user emotions (such as generating emotion-related audio, providing alerts for abnormal emotions, and processing emotion-related images). However, the computational and storage resource requirements of the same emotion recognition model vary when running on electronic devices with different hardware configurations. For devices with lower hardware specifications, the installation package size of some models may be too large, making installation impossible or difficult to complete even if successful. While related technologies typically deploy emotion recognition models in the cloud, this approach may compromise user privacy.

[0003] Therefore, how to achieve efficient local operation of emotion recognition by being compatible with electronic devices with different hardware configurations has become an urgent technical problem to be solved. Summary of the Invention

[0004] The main objective of this application is to propose an emotion recognition method, electronic device, and storage medium, which are designed to be compatible with electronic devices with different hardware configurations, so as to achieve efficient local operation of emotion recognition.

[0005] To achieve the above objectives, a first aspect of this application proposes an emotion recognition method, the method comprising:

[0006] Obtain the original emotion recognition model and the processor frequency and processor bit width of the preset electronic device;

[0007] The computing power of the preset electronic device is evaluated based on the processor frequency to obtain the computing power level of the preset electronic device;

[0008] The original emotion recognition model is segmented to obtain at least two intermediate model segments;

[0009] Calculate the target sparsity of each intermediate model segment based on the computing power level;

[0010] For each intermediate model segment, the intermediate model segment is compressed according to the target sparsity to obtain a compressed model segment; wherein, the compressed model segment has compressed network weights;

[0011] For each compressed model segment, the weights of the compressed network are grouped according to the processing bit width of the processor to obtain weight groups, and the weight groups are encapsulated into target model segments;

[0012] The current model segment is determined from each of the target model segments, and the current model segment is loaded into the shared memory of the preset electronic device. Emotion recognition is performed based on a preset face image using the current model segment located in the shared memory, and the previously loaded model segment is removed from the shared memory during the emotion recognition process.

[0013] In some embodiments, grouping the compressed network weights according to the processor's processing bit width to obtain weight groups includes:

[0014] The reference weight quantity value is determined based on the processor processing bit width, and the compressed network weights are grouped based on the reference weight quantity value to obtain candidate weight reorganization;

[0015] Quantization thresholds are calculated based on the maximum and minimum values ​​in each candidate weight reorganization and the number of reference weights.

[0016] The candidate weight reorganization is linearly quantized according to the quantization threshold to obtain the weight grouping.

[0017] In some embodiments, determining the reference weight quantity value based on the processor processing bit width includes:

[0018] Obtain the remaining battery power and processor operating temperature of the preset electronic device;

[0019] The data bit width is determined based on the remaining battery power and the processor operating temperature.

[0020] The reference weight quantity value is calculated based on the processor processing bit width and the data bit width.

[0021] In some embodiments, the intermediate model segment includes multiple output channels, each output channel having channel network weights. The step of compressing the intermediate model segment according to the target sparsity to obtain a compressed model segment includes:

[0022] For each output channel, the importance score of the output channel is calculated based on the channel network weights;

[0023] Based on the target sparsity and the importance score, channel pruning is performed on the intermediate model segment to obtain the compressed model segment.

[0024] In some embodiments, the step of performing channel pruning on the intermediate model segment based on the target sparsity rate and the importance score to obtain the compressed model segment includes:

[0025] The number of channels to be retained is calculated based on the target sparsity and the number of output channels;

[0026] The output channels are filtered based on the number of channels to be retained and the importance score to obtain the target channels;

[0027] The target channels are sequentially identified, and the compressed model segment is determined based on the sequentially identified target channels.

[0028] In some embodiments, calculating the target sparsity of each intermediate model segment based on the computing power level includes:

[0029] Obtain the baseline sparsity for each of the intermediate model segments;

[0030] Obtain the computing power adjustment factor based on the aforementioned computing power level;

[0031] The decay factor of the baseline sparsity is calculated based on the computing power adjustment factor and the computing power level;

[0032] For each intermediate model segment, the attenuation factor and the baseline sparsity of the intermediate model segment are multiplied to obtain the target sparsity of the intermediate model segment.

[0033] In some embodiments, the step of evaluating the computing power of the preset electronic device based on the processor frequency to obtain the computing power level of the preset electronic device includes:

[0034] The throughput of the processor of the preset electronic device is obtained by measuring the speed of the processor based on a preset convolution operator.

[0035] A computing power evaluation score is obtained by evaluating the computing power based on the processor frequency and the throughput.

[0036] The computing power level is obtained by querying the preset score level mapping relationship based on the computing power evaluation score.

[0037] In some embodiments, after performing a computing power assessment based on the processor frequency and the throughput to obtain a computing power assessment score, the method further includes:

[0038] Obtain the processor model of the preset electronic device;

[0039] The baseline level is obtained by querying the preset model level mapping relationship based on the processor model.

[0040] The computing power assessment score is updated based on the benchmark level.

[0041] To achieve the above objectives, a second aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method of the first aspect described above.

[0042] To achieve the above objectives, a third aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first aspect described above.

[0043] The emotion recognition method, electronic device, and storage medium proposed in this application evaluate the computing power of the processor frequency of a preset electronic device to obtain a computing power level. Based on the computing power level, the original emotion recognition model is segmented and differentially compressed. At the same time, the weights of the compressed network are grouped and encapsulated based on the processor's processing bit width. The model inference is executed by loading and replacing segments in shared memory. This allows the original emotion recognition model to be stored and adapted according to the hardware capabilities of the preset electronic device. Thus, while ensuring the realization of the emotion recognition function, the overall storage occupation and runtime memory pressure of the model are reduced, the local operating efficiency of low computing power devices is improved, and the privacy leakage risk caused by cloud deployment is avoided. Ultimately, the compatibility and stability of the emotion recognition model on electronic devices with different hardware configurations are improved. Attached Figure Description

[0044] Figure 1 This is a flowchart of the emotion recognition method provided in the embodiments of this application;

[0045] Figure 2 yes Figure 1 The flowchart of step S102 in the document;

[0046] Figure 3 yes Figure 1 The flowchart of step S104 in the process;

[0047] Figure 4 yes Figure 1 The flowchart of step S105 in the process;

[0048] Figure 5 yes Figure 4 The flowchart of step S402 in the document;

[0049] Figure 6 yes Figure 1 The flowchart of step S106 in the process;

[0050] Figure 7 yes Figure 6 The flowchart of step S601 in the process;

[0051] Figure 8This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] First, let's analyze some of the terms used in this application:

[0056] Sparsity ratio refers to the proportion of parameters or channels that are removed or zeroed out during model compression or pruning, representing the sparsity of the model. A higher sparsity ratio indicates that more computational units are cut, effectively reducing the model's computational complexity and storage overhead, but it may affect the model's accuracy to some extent.

[0057] Model Segment: refers to a sub-model unit obtained by dividing the model based on model depth or network structure. Each model segment is usually composed of several consecutive network layers and can be distinguished by segment identifiers to support segmented processing, segmented loading, or differentiated optimization.

[0058] Vector Processing Width: This parameter represents the number of bits of data that a processor can process in parallel during a single computation. It is usually preset by the hardware architecture and, together with the data bit width, determines the scale of data that can be processed in parallel at one time, thus affecting the overall computational efficiency and throughput.

[0059] To meet users' demands for intelligent features on electronic devices, most mobile applications implement emotion recognition based on facial images using artificial intelligence models. This allows for different intelligent functions based on user emotions (such as generating emotion-related audio, issuing alerts for abnormal emotions, and performing emotion-related image processing). However, the computational and storage resource requirements of the same emotion recognition model vary when running on electronic devices with different hardware configurations. For devices with lower hardware configurations, the installation package of some models may be too large, making installation impossible or difficult to complete even if successful. In related technologies, emotion recognition models are typically deployed in the cloud, but this approach may compromise user privacy. Other related technologies forcibly degrade model performance for devices with lower hardware configurations; while this allows the model to run smoothly on such devices, it significantly weakens the user experience.

[0060] Therefore, how to achieve efficient local operation of emotion recognition by being compatible with electronic devices with different hardware configurations has become an urgent technical problem to be solved.

[0061] Based on this, embodiments of this application provide an emotion recognition method, an electronic device, and a storage medium, which are designed to be compatible with electronic devices with different hardware configurations in order to achieve efficient local operation of emotion recognition.

[0062] The emotion recognition method, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the emotion recognition method in this application is described.

[0063] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0064] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0065] The emotion recognition method provided in this application relates to the field of artificial intelligence technology. The emotion recognition method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the emotion recognition method, but is not limited to the above forms.

[0066] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0067] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0068] Figure 1 This is an optional flowchart of the emotion recognition method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0069] Step S101: Obtain the original emotion recognition model and the processor frequency and processor processing bit width of the preset electronic device;

[0070] Step S102: Evaluate the computing power of the preset electronic device based on the processor frequency to obtain the computing power level of the preset electronic device;

[0071] Step S103: Perform model segmentation on the original emotion recognition model to obtain at least two intermediate model segments;

[0072] Step S104: Calculate the target sparsity of each intermediate model segment based on the computing power level;

[0073] Step S105: For each intermediate model segment, compress the intermediate model segment according to the target sparsity to obtain a compressed model segment; wherein, the compressed model segment has compressed network weights.

[0074] Step S106: For each compressed model segment, the weights of the compressed network are grouped according to the processor's processing bit width to obtain weight groups, and the weight groups are encapsulated into target model segments;

[0075] Step S107: Determine the current model segment from each target model segment, load the current model segment into the shared memory of the preset electronic device, perform emotion recognition based on the preset face image using the current model segment located in the shared memory, and remove the previously loaded model segment from the shared memory during the emotion recognition process.

[0076] Steps S101 to S107 as illustrated in this embodiment assess the computing power of the processor frequency of a preset electronic device and obtain its computing power level. The original emotion recognition model is then segmented and differentially compressed based on this computing power level. Simultaneously, the weights of the compressed network are grouped and encapsulated based on the processor's processing bit width. Model inference is then executed by loading and replacing segments in shared memory. This allows the original emotion recognition model to be stored and adapted to the hardware capabilities of the preset electronic device. This reduces the overall storage footprint and runtime memory pressure of the model while ensuring the realization of the emotion recognition function, improving the local operating efficiency of low-computing-power devices. It also avoids the privacy leakage risks associated with cloud deployment, ultimately achieving compatibility of the emotion recognition model with electronic devices of different hardware configurations, thus enabling efficient local operation of emotion recognition.

[0077] In step S101 of some embodiments, the original emotion recognition model can be a neural network model trained by supervised learning, used to classify emotions in the input face image. The original emotion recognition model can be a three-dimensional convolutional neural network model (3D-CNN).

[0078] A preset electronic device is a terminal device used to run a compressed, lightweight model to achieve emotion recognition. In some embodiments, the preset electronic device may also have image acquisition and processing capabilities simultaneously. Preset electronic devices include, but are not limited to, smartphones, tablets, or embedded devices. Processor frequency refers to the number of clock cycles per unit time of the processor in the preset electronic device, and processor processing bit width refers to the number of bits of data that the processor in the preset electronic device can process in a single operation, i.e., the vector processing bit width. Hardware parameter information of the preset electronic device can be read through the device system's low-level interface, or processor specification information can be queried through the device driver. For example, the processor frequency can be obtained through the hardware information interface provided by the operating system, and the processor processing bit width can be determined through the processor instruction set description information. For example, in this embodiment, the preset electronic device is a mobile phone, and the processor frequency is read as 3300MHz and the processor processing bit width as 128bit through the hardware system interface.

[0079] In step S102 of some embodiments, the computing power level is used to characterize the overall computing power of a preset electronic device when performing a model inference task. Specifically, please refer to... Figure 2 In some embodiments, step S102 may include, but is not limited to, steps S201 to S203:

[0080] Step S201: The processor of the preset electronic device is measured based on the preset convolution operator to obtain the throughput of the processor;

[0081] Step S202: Perform a computing power evaluation based on the processor frequency and throughput to obtain a computing power evaluation score;

[0082] Step S203: Query the preset score level mapping relationship based on the computing power evaluation score to obtain the computing power level.

[0083] In step S201 of some embodiments, the convolution operator is a standardized computational unit used to simulate the convolutional layer computation process in an emotion recognition model. It performs fixed-scale convolution operations on the input data to reflect the processor's computational performance in actual model inference. Throughput refers to the processor's data processing capability or the number of operations it can perform on convolution operations per unit time. In this embodiment, the convolution operator is a 3×3 convolution kernel, and the input feature map has a size of 1×224×224×3. The processor's computational speed is tested through this convolution operation. Performing multiple rounds of convolution operations on the same input data, completing 42 billion operations in 1 second, corresponds to a throughput of 42 GOP / s.

[0084] In step S202 of some embodiments, the computing power evaluation score is a quantitative indicator used to comprehensively characterize the processor's computing capabilities. Specifically, the computing power evaluation score is calculated by weighting the processor frequency and throughput using preset weighting coefficients. The calculation method for the computing power evaluation can refer to the following analytical formula: (1),

[0085] in, This represents the computing power assessment score. In this embodiment, the weighting coefficient representing the processor frequency is... The value can be 0.5. The weighting coefficient representing throughput, in this embodiment, The value can be 0.5. This indicates the processor frequency. Indicates throughput.

[0086] In other embodiments, the computing power evaluation process may also consider the processor model of the preset electronic device. Specifically, after step S202, the emotion recognition method provided in this application embodiment includes: 1. Obtaining the processor model of the preset electronic device. The hardware information query interface provided by the operating system is called to read the processor model identifier string from the device's system information file or processor identifier register, or the processor model parameter is obtained through the interface provided by the device driver layer. 2. Querying the preset model level mapping relationship according to the processor model to obtain the baseline level. The model level mapping relationship refers to the preset correspondence rule that maps different processor models to corresponding performance levels. After obtaining the processor model, a query is performed in the corresponding table through string matching or key-value lookup to obtain the corresponding baseline level. The baseline level refers to the performance level identifier pre-divided based on the processor model, used to initially classify the computing power of the preset electronic device before actual speed testing or computing power evaluation. In this embodiment, the numerical range of the baseline level is [0,9], and it is an integer. 3. Updating the computing power evaluation score according to the baseline level. Specifically, the update process can refer to the following analytical expression:

[0087] (2),

[0088] in, This represents the computing power assessment score. In this embodiment, the weighting coefficient representing the processor frequency is... The value can be 0.2. The weighting coefficient representing throughput, in this embodiment, The value can be 0.3. The weighting coefficient representing throughput, in this embodiment, The value can be 0.5. This indicates the processor frequency. Indicates throughput. Indicates the baseline level.

[0089] Through the three steps described above, data for evaluating computing power is obtained from three dimensions: hardware model, CPU frequency, and actual throughput measurement, avoiding misjudgment based on a single indicator. For example, in processors of the same series of signals, by limiting the number of computing units in the neural network processing unit, reducing the operating frequency, or disabling some functional modules, the actual computing power of the neural network processing unit can be made lower than the design state of the full configuration version.

[0090] In step S203 of some embodiments, the score-level mapping relationship is a preset correspondence rule used to convert computing power evaluation scores into discrete computing power levels. The score-level mapping relationship can be established by using an interval division method to establish the mapping between different score intervals and corresponding computing power levels.

[0091] In this embodiment, the numerical range of the computing power level identifier is [0, 9], and it is an integer. The calculated computing power evaluation score is matched with multiple preset score intervals, and the computing power level identifier corresponding to the interval is output. For example, the preset baseline level of the electronic device is 8, the processor frequency is 1.1 GHz, and the throughput is 42 GOP / s. The computing power evaluation score calculated according to the analytical formula (2) is 6.7. When it falls into the interval [6, 7], the computing power level can be determined to be 7.

[0092] Steps S201 to S203 as shown in the embodiments of this application obtain processor throughput by introducing a processor speed measurement process based on convolution operators, and jointly evaluate processor frequency and throughput to form a computing power evaluation score. Then, the computing power level is determined through the score level mapping relationship, so that the computing power level can comprehensively reflect the processor's computing power in the actual model inference process, thereby improving the accuracy and stability of computing power evaluation results. This provides a more reliable basis for the subsequent adaptation of the emotion recognition model, which is conducive to improving the running efficiency of the model in different hardware environments and reducing resource waste.

[0093] In step S103 of some embodiments, the intermediate model segment refers to the model segment obtained by dividing the original emotion recognition model according to the network depth. Each intermediate model segment contains a portion of the continuous network layers in the original emotion recognition model. In this embodiment, the original emotion recognition model is divided into 5 segments according to depth, resulting in a total of 5 intermediate model segments. A segment number identifier is added to each intermediate model segment. The segment number identifiers are set sequentially from the input layer to the output layer, with the intermediate model segments closer to the input layer corresponding to smaller segment number identifiers and the intermediate model segments closer to the output layer corresponding to larger segment number identifiers.

[0094] In step S104 of some embodiments, the target sparsity can be used to characterize the proportion of output channels that are pruned in the intermediate model segment. For the calculation process of the target sparsity, please refer to [link to relevant documentation]. Figure 3 In some embodiments, step S104 may include, but is not limited to, steps S301 to S304:

[0095] Step S301: Obtain the baseline sparsity of each intermediate model segment;

[0096] Step S302: Obtain the computing power adjustment factor based on the computing power level;

[0097] Step S303: Calculate the attenuation factor of the baseline sparsity based on the computing power adjustment factor and computing power level;

[0098] Step S304: For each intermediate model segment, multiply the attenuation factor and the baseline sparsity of the intermediate model segment to obtain the target sparsity of the intermediate model segment.

[0099] In step S301 of some embodiments, the baseline sparsity rate is used to characterize the pruning ratio of each intermediate model segment without considering computational constraints, reflecting the tolerance of the intermediate model segment to parameter compression. Its value is related to the functional position of the intermediate model segment in the original emotion recognition model. In this embodiment, the baseline sparsity rate of each intermediate model segment is preset, and the specific correspondence between the model segment number identifier, the model segment position, and the baseline sparsity rate value can be referred to Table 1. The bottom layer is the network layer closest to the input layer, and the top layer is the network layer closest to the output layer.

[0100] Table 1

[0101]

[0102] In step S302 of some embodiments, the computing power adjustment factor is a control parameter used to adjust the magnitude of the change in the baseline sparsity rate according to the computing power level, reflecting the degree of influence of computing power changes on pruning intensity. In this embodiment, when the computing power level is [0,3), the value of the computing power adjustment factor is 0.05. When the computing power level is [3,6), the value of the computing power adjustment factor is 0.10. When the computing power level is [6,9], the value of the computing power adjustment factor is 0.15.

[0103] In other embodiments, the computing power adjustment factor can be determined based on the accuracy requirements of the emotion recognition application scenario. For example, in accuracy-priority application scenarios, the computing power adjustment factor can be set to 0.05; in general application scenarios, it can be set to 0.10; and in performance-priority application scenarios, it can be set to 0.15.

[0104] In step S303 of some embodiments, the attenuation factor is a scaling factor used to dynamically adjust the reference sparsity, and its calculation process can refer to the following analytical formula:

[0105] (3),

[0106] in, Indicates the attenuation factor. This represents the computing power adjustment factor. Indicates the computing power level.

[0107] In step S304 of some embodiments, the target sparsity ratio is the actual pruning ratio used to guide the intermediate model segments in performing emotion recognition model pruning. The target sparsity ratio can be calculated analytically using the following formula:

[0108] (4),

[0109] Where i represents the segment identifier of the intermediate model segment. This represents the target sparsity of the i-th intermediate model segment applied to the preset electronic device. This represents the baseline sparsity of the i-th intermediate model segment. This represents the attenuation factor.

[0110] Steps S301 to S304 as shown in the embodiments of this application pre-set a baseline sparsity for different intermediate model segments and introduce a computing power adjustment factor in combination with the computing power level to attenuate and adjust the baseline sparsity, thereby forming a target sparsity that matches the computing power level of electronic devices. This enables each intermediate model segment to have differentiated compression capabilities during the pruning process. While ensuring the ability to express key features of the emotion recognition model, it effectively reduces the model parameter scale and computational complexity, improves the model's operating efficiency and stability on low computing power devices, and enhances the adaptability and deployment flexibility of the emotion recognition model in different hardware environments.

[0111] In step S105 of some embodiments, the intermediate model segment includes multiple output channels, each with channel network weights. The compressed model segment refers to the model structure obtained after pruning the intermediate model segment using the emotion recognition model; the compressed network weights can be the weight parameters retained during the pruning process. See also... Figure 4 In some embodiments, step S105 may include, but is not limited to, steps S401 to S402:

[0112] Step S401: For each output channel, calculate the importance score of the output channel based on the channel network weights;

[0113] Step S402: Perform channel pruning on the intermediate model segment based on the target sparsity and importance score to obtain the compressed model segment.

[0114] In step S401 of some embodiments, the importance score is a quantitative indicator used to characterize the contribution of the output channel to feature expression in the emotion recognition model. The importance score can be the L1 norm or L2 norm of the channel network weights of the corresponding output channel. In this embodiment, the importance score is preferably the L1 norm because it requires less computation. The larger the value of the L1 norm, the higher the contribution of the output channel in the feature extraction process.

[0115] In step S402 of some embodiments, please refer to Figure 5 Step S402 may also include, but is not limited to, steps S501 to S503:

[0116] Step S501: Calculate the number of channels to be retained based on the target sparsity and the number of output channels;

[0117] Step S502: Filter the output channels according to the number of channels to be retained and the importance score to obtain the target channels;

[0118] Step S503: Sequentially identify the target channels and determine the compression model segment based on the sequentially identified target channels.

[0119] In step S501 of some embodiments, the number of channels to be retained refers to the number of output channels that need to be retained during the channel pruning process. The number of channels to be retained is obtained by multiplying the number of output channels by the retention ratio, where the retention ratio is (1 - target sparsity), and the calculation result is rounded. For example, when the number of output channels is 256 and the target sparsity is 0.4, the number of channels to be retained can be calculated to be 154.

[0120] In step S502 of some embodiments, the target channel refers to the output channel that is retained for constructing the compressed model segment after being filtered according to its importance score. First, all output channels are sorted based on their importance scores to obtain a channel index sequence arranged in descending order of importance scores. Then, the output channel that is within the range of the number of channels to be retained in the sorting result is selected as the target channel.

[0121] Furthermore, a pruning mask can be generated to prune channels in the intermediate model segment. For the target channel, the mask value at the corresponding position is set to 1, and for the unselected output channel, the mask value at the corresponding position is set to 0. The pruning mask is applied to the channel network weights of the intermediate model segment. By masking or deleting the channel network weights corresponding to the output channel with a mask value of 0, a channel structure containing only the target channel is obtained. Based on the retained target channel, the network structure of the intermediate model segment is reconstructed, thereby forming a compressed model segment.

[0122] In step S503 of some embodiments, sequentially identifying the target channels refers to reassigning continuous indices to the selected target channels according to predetermined rules to form a structurally continuous channel arrangement. It is important to note that the order here is not based on importance scores, but rather on the original order of the output channels in the intermediate model segment, ensuring that the corresponding channel identifiers are numerically continuous. The importance score sorting is merely used to determine which channels need to be retained. Further, assume the intermediate model segment has 6 output channels [1,2,3,4,5,6]. Given a target sparsity rate of 50%, an importance score is calculated for each output channel, and the channels corresponding to the three highest importance scores are determined to be channel 1, channel 4, and channel 6. Channels 1, 4, and 6 are retained, leaving the target channel set [1,4,6]. In step S503, these channels are then re-labeled sequentially to obtain the target channel set [1,2,3], and the re-labeled target channel set is used as the compressed model segment.

[0123] Re-identifying the pruned channels with consecutive serial numbers is to ensure that the channels in the compressed model segment form a continuous arrangement in the storage space, thereby aligning with the fixed-width vector register. This enables the simultaneous parallel loading of multiple channel weights under INT4 quantization, reducing memory access overhead caused by discrete accesses, avoiding multiple loading and mask splicing operations due to discontinuous channel distribution, improving the data utilization of the vector computing unit, reducing memory access frequency and bandwidth waste, and thus significantly improving the execution efficiency and energy efficiency of the emotion recognition model on mobile processors.

[0124] Steps S501 to S503 as shown in the embodiments of this application determine the scale of output channels to be retained based on the target sparsity rate, and sort and filter the output channels in combination with the importance score. The selected target channels are further sequentially identified and reconstructed, so that the compressed model segment maintains structural continuity and data access consistency after channel pruning. This reduces the model parameter scale and computational overhead while ensuring the stability of the emotion recognition model inference process, improving the execution efficiency and deployment reliability of the pruned model, and enhancing the adaptability of the emotion recognition model in resource-constrained electronic devices.

[0125] Steps S401 to S402 as illustrated in this embodiment calculate the importance score for each output channel in the intermediate model segment and selectively retain or delete output channels based on the target sparsity rate. This allows output channels with higher importance scores to be retained while those with lower importance scores to be pruned. In this way, while reducing the model parameter size and computational complexity, the feature expression ability that has a key impact on the emotion recognition result is effectively preserved, improving the accuracy and stability of the emotion recognition model after pruning. This achieves a balance between model compression efficiency and recognition performance, and enhances the deployment feasibility and operational reliability of the emotion recognition model in resource-constrained electronic devices.

[0126] In step S106 of some embodiments, grouping the compressed network weights according to the processor's processing bit width can be done by dividing the compressed network weights according to the data bit width supported by the processor in a single operation. Each set of weight parameters is a weight group, thereby ensuring that each weight group is aligned with the processor's data path during loading and operation. Subsequently, for each weight group, it is packaged together with the segment number and size information of its corresponding compressed model segment and encapsulated into the installation data package (such as Android Application Package assets) corresponding to the compressed model segment.

[0127] Further, please refer to Figure 6 In some embodiments, step S106 includes, but is not limited to, steps S601 to S603:

[0128] Step S601: Determine the reference weight quantity value according to the processor processing bit width, and group the compressed network weights based on the reference weight quantity value to obtain candidate weight reorganization;

[0129] Step S602: Calculate the quantization threshold based on the maximum value, minimum value, and reference weight quantity value in each candidate weight reorganization;

[0130] Step S603: Perform linear quantization on the candidate weight reorganization according to the quantization threshold to obtain weight grouping.

[0131] In step S601 of some embodiments, the reference weight quantity value is a grouping benchmark parameter used to determine the number of weights contained in each group of weights. The reference weight quantity value is set corresponding to the processor's processing bit width to ensure that the grouped weight data meets hardware alignment requirements during storage and computation. Candidate weight reassembly refers to the grouping result of the original compressed network weights. The compressed network weights are sequentially divided according to the reference weight quantity value, and each consecutive reference weight quantity value of weights constitutes a candidate weight reassembly. For example, when the processor's processing bit width is 128 bits and INT4 quantization is used, the reference weight quantity value can be determined to be 32. The compressed network weights are divided into candidate weight reassemblies of every 32 weights, thereby obtaining multiple candidate weight reassemblies.

[0132] Further, please refer to Figure 7 In some embodiments, step S601 may include, but is not limited to, steps S701 to S703:

[0133] Step S701: Obtain the remaining battery power and processor operating temperature of the preset electronic device;

[0134] Step S702: Determine the data bit width based on the remaining battery power and the processor operating temperature;

[0135] Step S703: Calculate the reference weight quantity value based on the processor processing bit width and data bit width.

[0136] In step S701 of some embodiments, the remaining battery power and processor operating temperature are key parameters used to reflect the current operating state of the preset electronic device. The current battery percentage information can be read from the battery management module, and the temperature value can be read from the system node corresponding to the processor temperature sensor, by calling the hardware status monitoring interface provided by the operating system.

[0137] In step S702 of some embodiments, the data bit width refers to the number of binary bits used to represent a single weight value, used to describe the numerical precision of model parameters during storage and calculation. A pre-established data bit width selection rule corresponding to the remaining battery power and processor operating temperature is used. When the remaining battery power is below a preset threshold and the processor operating temperature is above a preset temperature threshold, a lower data bit width is selected; otherwise, a higher data bit width is selected. For example, when the remaining battery power is less than 15% and the processor operating temperature is above 43°C, the data bit width is set to INT4; when the remaining battery power is greater than or equal to 15% or the processor operating temperature is not higher than 43°C, the data bit width is set to INT8. Therefore, the number of reference weight values ​​is an integer multiple of the processor processing bit width.

[0138] In step S703 of some embodiments, the processor processing bit width is divided by the data bit width, and the result is used as a reference weight value. For example, when the processor processing bit width is 128 bits and the data bit width is INT4, the reference weight value can be calculated as 128 / 4 = 32.

[0139] Steps S701 to S703 as illustrated in this application embodiment introduce the remaining battery power and processor operating temperature as constraints to dynamically adjust the data bit width, enabling the data bit width to adapt to changes in device operating status. Based on this, the reference weight quantity value is calculated in conjunction with the processor processing bit width, so that the weight grouping scale meets hardware alignment requirements while taking into account power consumption and thermal management needs. Thus, while ensuring the basic performance of the emotion recognition model, energy consumption and thermal load are effectively reduced, the stability and battery life of electronic devices under different operating states are improved, and the deployment adaptability and operational reliability of the emotion recognition model in actual use environments are enhanced.

[0140] In step S602 of some embodiments, the quantization threshold is a scale parameter used to map floating-point weights to a discrete integer value range. It is calculated by obtaining the maximum and minimum values ​​in each candidate weight reorganization and obtaining the quantization step size by dividing the difference between the maximum and minimum values ​​by the number of quantization level intervals.

[0141] For example, for a set of candidate weight reorganizations, the original FP16 weights are [0.023, -0.156, 0.089, -0.034, 0.201, -0.118, 0.067, -0.092, ...] with a total of 32 weights. The maximum value within the group is 0.201, and the minimum value within the group is -0.156. Under the condition that the INT4 quantization range is -8 to 7, there are a total of 16 discrete values, which correspond to 15 intervals. Therefore, the quantization threshold can be calculated as (0.201-(-0.156)) / 15≈0.0238.

[0142] In step S603 of some embodiments, each floating-point weight is subtracted from its minimum value and then divided by the quantization threshold to obtain the corresponding quantization index. The quantization index is then rounded, and the result is limited to a preset integer range to obtain the corresponding low-bit integer representation. Following the example of step S602, the weights in the candidate weight reorganization are quantized according to the aforementioned quantization threshold, resulting in INT4 quantization values ​​of [1, -6, 4, -1, 8, -5, 3, -4, ...]. Adjacent INT4 values ​​are then packaged and stored according to their high 4 bits and low 4 bits to form byte data, such as 0x1A and 0x4F.

[0143] Steps S601 to S603 of this embodiment determine the number of reference weights based on the processor's processing bit width, matching the group size of the candidate weight group with the width of the processor's vector register. A quantization threshold is calculated based on the numerical distribution of the candidate weight groups to perform linear quantization, thereby converting the compressed network weights into a low-bit representation and storing them compactly. This significantly compresses the model size at the storage level and aligns it with the vectorized computation unit at the computation level, allowing weight data to be loaded into the processor register in a single group. This reduces performance loss from discrete memory access operations, lowers memory bandwidth usage, and reduces data transmission latency. Furthermore, by combining the continuous channel arrangement structure, the weight data remains continuously distributed in the storage space, improving vector loading efficiency and data utilization. This significantly enhances the loading speed and inference efficiency of the emotion recognition model on electronic devices and strengthens its operational stability and deployment feasibility in low-bandwidth environments.

[0144] In step S107 of some embodiments, the face image is image data input by the user to a preset electronic device. The current model segment can be the target model segment scheduled for execution during inference, and the shared memory can be a storage area in the preset electronic device for shared access by multiple computing units. Performing emotion recognition based on the preset face image using the current model segment located in the shared memory can be achieved by loading the current model segment into the shared memory and then calling the processor to perform forward inference computation on the input face image. Removing the previously loaded model segment from the shared memory during emotion recognition can be achieved by releasing the data corresponding to the previous model segment from the shared memory before loading a new current model segment, thereby reducing memory usage. For example, if a first target model segment is loaded into the shared memory, then the first target model segment becomes the current model segment, and feature extraction is performed on the face image. Before the first target model segment completes inference, the second target model segment can be loaded. After the first target model segment completes inference, the second target model segment is used as the current model segment, and further inference is performed on the output features of the first target model segment. At the same time, the first target model segment is removed from the shared memory. This process continues until all target model segments are loaded into the shared memory and emotion recognition is completed, and finally the emotion recognition result is output.

[0145] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned emotion recognition method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0146] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0147] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0148] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the emotion recognition method of the embodiments of this application.

[0149] The 803 input / output interface is used to implement information input and output.

[0150] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0151] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0152] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0153] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described emotion recognition method.

[0154] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0155] The emotion recognition method, electronic device, and storage medium provided in this application evaluate the computing power of the processor frequency of a preset electronic device to obtain a computing power level. Based on the computing power level, the original emotion recognition model is segmented and differentially compressed. At the same time, the weights of the compressed network are grouped and encapsulated based on the processor's processing bit width. The model inference is executed by segment loading and replacement through shared memory. This allows the original emotion recognition model to be stored and adapted according to the hardware capabilities of the preset electronic device. Thus, while ensuring the realization of the emotion recognition function, the overall storage occupation and runtime memory pressure of the model are reduced, the local operating efficiency of low computing power devices is improved, and the privacy leakage risk caused by cloud deployment is avoided. Ultimately, the compatibility and stability of the emotion recognition model on electronic devices with different hardware configurations are improved.

[0156] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0157] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0158] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0160] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0161] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0163] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An emotion recognition method, characterized in that, The method includes: Obtain the original emotion recognition model and the processor frequency and processor bit width of the preset electronic device; The computing power of the preset electronic device is evaluated based on the processor frequency to obtain the computing power level of the preset electronic device; The original emotion recognition model is segmented to obtain at least two intermediate model segments; Calculate the target sparsity of each intermediate model segment based on the computing power level; For each intermediate model segment, the intermediate model segment is compressed according to the target sparsity to obtain a compressed model segment; wherein, the compressed model segment has compressed network weights; For each compressed model segment, the compressed network weights are grouped according to the processor's processing bit width to obtain weight groups, and the weight groups are encapsulated into target model segments; wherein, the step of grouping the compressed network weights according to the processor's processing bit width to obtain weight groups includes: determining a reference weight quantity value according to the processor's processing bit width, and grouping the compressed network weights based on the reference weight quantity value to obtain candidate weight reassemblies; calculating a quantization threshold based on the maximum value, minimum value, and reference weight quantity value in each candidate weight reassembly; and performing linear quantization on the candidate weight reassemblies according to the quantization threshold to obtain the weight groups; The current model segment is determined from each of the target model segments, and the current model segment is loaded into the shared memory of the preset electronic device. Emotion recognition is performed based on a preset face image using the current model segment located in the shared memory, and the previously loaded model segment is removed from the shared memory during the emotion recognition process.

2. The method according to claim 1, characterized in that, The step of determining the reference weight quantity value based on the processor processing bit width includes: Obtain the remaining battery power and processor operating temperature of the preset electronic device; The data bit width is determined based on the remaining battery power and the processor operating temperature. The reference weight quantity value is calculated based on the processor processing bit width and the data bit width.

3. The method according to claim 1, characterized in that, The intermediate model segment includes multiple output channels, each output channel having channel network weights. The step of compressing the intermediate model segment according to the target sparsity to obtain a compressed model segment includes: For each output channel, the importance score of the output channel is calculated based on the channel network weights; Based on the target sparsity and the importance score, channel pruning is performed on the intermediate model segment to obtain the compressed model segment.

4. The method according to claim 3, characterized in that, The step of performing channel pruning on the intermediate model segment based on the target sparsity rate and the importance score to obtain the compressed model segment includes: The number of channels to be retained is calculated based on the target sparsity and the number of output channels; The output channels are filtered based on the number of channels to be retained and the importance score to obtain the target channels; The target channels are sequentially identified, and the compressed model segment is determined based on the sequentially identified target channels.

5. The method according to any one of claims 1 to 4, characterized in that, The step of calculating the target sparsity of each intermediate model segment based on the computing power level includes: Obtain the baseline sparsity for each of the intermediate model segments; Obtain the computing power adjustment factor based on the aforementioned computing power level; The decay factor of the baseline sparsity is calculated based on the computing power adjustment factor and the computing power level; For each intermediate model segment, the attenuation factor and the baseline sparsity of the intermediate model segment are multiplied to obtain the target sparsity of the intermediate model segment.

6. The method according to any one of claims 1 to 4, characterized in that, The step of evaluating the computing power of the preset electronic device based on the processor frequency to obtain the computing power level of the preset electronic device includes: The throughput of the processor of the preset electronic device is obtained by measuring the speed of the processor based on a preset convolution operator. A computing power evaluation score is obtained by evaluating the computing power based on the processor frequency and the throughput. The computing power level is obtained by querying the preset score level mapping relationship based on the computing power evaluation score.

7. The method according to claim 6, characterized in that, After performing a computing power assessment based on the processor frequency and the throughput to obtain a computing power assessment score, the method further includes: Obtain the processor model of the preset electronic device; The baseline level is obtained by querying the preset model level mapping relationship based on the processor model. The computing power assessment score is updated based on the benchmark level.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images