Feature analysis method for deep learning model, storage medium and electronic equipment

Abstract

The invention relates to the technical field of deep learning, and discloses a feature analysis method for a deep learning model, a storage medium and electronic equipment. The method comprises the steps of obtaining a mapping relationship between a neuron activation value of a deep learning model and a feature region of a classification object; determining a suspected feature region of the classification object according to the mapping relation; the suspected feature region represents a feature region which has high contribution to activation of a plurality of important neurons of the deep learning model at the same time; performing model training on the deep learning model based on the classification object to obtain an output data variable quantity; and determining a key feature region from the suspected feature regions according to the output data variable quantity. The pruning operation can be more targeted, and the scale and the calculation complexity of the model are reduced on the premise that the model performance is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, for example, to a feature analysis method, storage medium and electronic device for a deep learning model. Background Art

[0002] Some advanced deep neural network models, such as residual networks and BERT (Bidirectional Encoder Representations from Transformers), are highly praised for their excellent performance. However, deep learning models often have hundreds of millions of parameters and require huge computing resources and storage space, which makes them extremely difficult to deploy and run on resource-constrained devices (such as smart home appliances, smartphones, embedded systems, etc.).

[0003] In order to solve the above problems, weight pruning has emerged as an effective model compression technology. The core idea of ​​weight pruning is to reduce the complexity and computational complexity of the model by reducing unnecessary parameters (i.e. weights) in the model, while maintaining the performance of the model as much as possible. The implementation steps of weight pruning include: collecting weight data of each layer or each neuron connection during model training; setting thresholds for each layer or all connection weights, and sorting these weights by absolute values; deleting weights whose absolute values ​​are less than the threshold, that is, setting them to 0; retraining the remaining non-zero weights to obtain a more compact model.

[0004] Although weight pruning can significantly reduce the number of model parameters and the amount of computation, as well as memory usage, it is difficult to accurately assess the importance of weights. This is because some weights, although small in value, play an important role in the model, and deleting such weights will lead to a significant drop in model accuracy.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide a feature analysis method, storage medium, and electronic device for a deep learning model, which can make pruning operations more targeted and reduce the scale and computational complexity of the model while ensuring the performance of the model.

[0008] In some embodiments, a feature analysis method for a deep learning model is provided, including: obtaining a mapping relationship between a neuron activation value of a deep learning model and a feature region of a classification object; determining a suspected feature region of the classification object based on the mapping relationship; the suspected feature region represents a feature region that simultaneously contributes highly to the activation of multiple important neurons of the deep learning model; training the deep learning model based on the classification object to obtain a change in output data; and determining a key feature region from the suspected feature region based on the change in output data.

[0009] Optionally, obtaining a mapping relationship between neuron activation values ​​of a deep learning model and feature areas of a classification object includes: determining a target neuron in the deep learning model; configuring the activation value of the target neuron to be 1 and configuring the activation values ​​of other neurons to be 0 to obtain a fine-tuned deep learning model; and obtaining a mapping relationship between neuron activation values ​​of the deep learning model and feature areas of a classification object based on the fine-tuned deep learning model.

[0010] Optionally, based on the fine-tuning of the deep learning model, a mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object is obtained, including: performing back propagation based on the fine-tuning of the deep learning model, back propagating from the target neuron to the input layer, and obtaining reference input data with the same dimension as the input layer; visualizing the reference input data to obtain the characteristic area of ​​the classification object corresponding to the target neuron activation value; replacing the target neuron to traverse each neuron in the deep learning model, and obtaining the characteristic area of ​​the classification object corresponding to each neuron activation value as the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object.

[0011] Optionally, the deep learning model is trained based on the classification object to obtain the output data change, including: inputting the classification object into the deep learning model for model training to obtain original output data; inputting the occluded classification object into the deep learning model for model training to obtain reference output data; the occluded classification object is obtained by occluding the local area of ​​the classification object by a sliding window; and obtaining the output data change based on the original output data and the reference output data.

[0012] Optionally, the occlusion classification object is obtained in the following manner: the window size and occlusion position of the sliding window are determined; the occlusion position represents the location of the occluded local area in the occlusion classification object; based on the window size and the occlusion position, the area to be occluded of the classification object is determined; the pixel value of the area to be occluded is configured as a pixel threshold to obtain the occlusion classification object.

[0013] Optionally, the occlusion classification object is input into the deep learning model for model training to obtain reference output data, including: controlling the sliding window to move sequentially within the classification object according to the set sliding direction and step threshold to obtain multiple occlusion classification objects; and inputting multiple occlusion classification objects into the deep learning model for model training respectively to obtain parameter output data corresponding to each of the multiple occlusion classification objects.

[0014] Optionally, based on the output data change, the key feature area is determined from the suspected feature area, including: obtaining the output data change of each reference output data and the original output data; taking the occluded position of the sliding window corresponding to the reference output data with the largest output data change as the target position; and taking the suspected feature area containing the target position as the key feature area.

[0015] Optionally, the feature analysis method also includes: pruning the deep learning model according to the key feature area; performing model training on the pruned deep learning model to obtain a target deep learning model.

[0016] In some embodiments, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is run, the feature analysis method for the deep learning model described in the above embodiment is executed.

[0017] In some embodiments, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the feature analysis method for a deep learning model described in the above embodiment through the computer program.

[0018] The feature analysis method, storage medium, and electronic device for a deep learning model provided by the embodiments of the present disclosure can achieve the following technical effects:

[0019] Through the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object, it is possible to know the area with a large contribution to the activation of important neurons and the characteristic area that has a large contribution to the activation of multiple important neurons at the same time, that is, the suspected characteristic area. If a characteristic area has a large contribution to the activation of multiple important neurons at the same time, then the characteristic area is very likely to have a large impact on the model output. Therefore, the embodiment of the present disclosure first obtains the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object to determine the suspected characteristic area of ​​the classification object according to the mapping relationship. Then, the deep learning model is trained based on the classification object to obtain the output data change, and the key characteristic area is determined from the suspected characteristic area according to the output data change, so as to achieve accurate positioning of the key characteristic area. In the process of adaptive pruning of the deep learning model, accurately determining the key characteristic area helps to determine which part of the classification object is critical to the model performance, making the pruning operation more targeted, thereby reducing the scale and computational complexity of the model while ensuring the model performance.

[0020] The model analysis method for deep learning models provided by the embodiments of the present disclosure can determine the feature areas that are critical to model performance, namely the key feature areas, from the perspective of input data. When the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object is known, the neurons related to the key feature area can be known, which provides a reliable basis for weight evaluation in the pruning process. Compared with the related art, the embodiments of the present disclosure provide a new method for evaluating the importance of weights to determine the key weights, i.e., the weights of the neuron connections related to the key feature areas, including those weights that have small values ​​but play an important connection role in the model. Therefore, during the pruning process, the decrease in model accuracy caused by the mistaken deletion of key weights can be avoided, and the model can be ensured to maintain good performance after pruning while reducing the scale and computational complexity of the model.

[0021] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0023] Figure 1 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a feature analysis method for a deep learning model provided by an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a feature analysis method for a deep learning model provided by another embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of a feature analysis method for a deep learning model provided by another embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram of a feature analysis method for a deep learning model provided by another embodiment of the present disclosure;

[0028] Figure 6 It is a schematic diagram of a feature analysis device for a deep learning model provided by an embodiment of the present disclosure.

[0029] Reference numerals:

[0030] 1. Electronic equipment; 10. Equipment body;

[0031] 60. Feature analysis device for deep learning model; 600. Processor; 601. Memory; 602. Communication interface; 603. Bus. DETAILED DESCRIPTION

[0032] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0033] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0034] Unless otherwise stated, the term "plurality" means two or more.

[0035] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0036] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0037] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0038] Combination Figure 1 As shown, an embodiment of the present disclosure provides an electronic device 1, including a device body 10 and a feature analysis device 60 for a deep learning model.

[0039] In the disclosed embodiment, the feature analysis device 60 for the deep learning model is installed in the device body 10. The installation relationship described here is not limited to the placement inside the device body 10, but also includes the installation connection with other components of the electronic device 1, including but not limited to physical connection, electrical connection or signal transmission connection, etc. It can be understood by those skilled in the art that the feature analysis device 60 for the deep learning model can be adapted to a feasible electronic device 1, thereby realizing other feasible embodiments.

[0040] In the disclosed embodiment, the electronic device 1 may not be limited to a PC (Personal Computer), a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing equipment, a smart dishwasher, a smart projection equipment, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart speaker, a smart fresh air equipment, a smart kitchen and bathroom equipment, a smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification equipment, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0041] Optionally, combined Figure 6 As shown, the feature analysis device 60 for the deep learning model includes a processor 600. The processor 600 can obtain the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object; can determine the suspected feature area of ​​the classification object according to the mapping relationship; can perform model training on the deep learning model based on the classification object to obtain the output data change amount; can determine the key feature area from the suspected feature area according to the output data change amount.

[0042] Combination Figure 1 The electronic device 1 shown in the present disclosure provides a model analysis method for a deep learning model, such as Figure 2 As shown, including:

[0043] S201, the processor obtains a mapping relationship between a neuron activation value of a deep learning model and a feature area of ​​a classification object.

[0044] In this step, the classification object represents the input data of the model. The classification object includes images or texts. When the classification object is an image, the deep learning model is used for image classification and recognition. When the classification object is text, the deep learning model is used for text classification and recognition. In the deep learning model, the neuron activation value reflects the degree of response of the model to the input data. By analyzing the activation of each neuron when the deep learning model processes the input data, a mapping relationship between the neuron activation value and the feature area of ​​the classification object is established to understand how the model focuses on different parts of the input data in the decision-making process, and how the model extracts features from the input data and classifies them, providing a basis for subsequent feature analysis.

[0045] S202: The processor determines a suspected feature region of the classification object according to the mapping relationship.

[0046] In this step, the mapping relationship refers to the mapping relationship between the neuron activation value of the deep learning model obtained in step S201 and the feature area of ​​the classification object. The suspected feature area represents a feature area that has a high contribution to the activation of multiple important neurons of the deep learning model at the same time. By determining the suspected feature area, it is preliminarily determined which areas may be important references for the model to make classification decisions, so as to narrow the scope of subsequent analysis.

[0047] S203: The processor performs model training on the deep learning model based on the classification object to obtain a change in output data.

[0048] S204: The processor determines a key feature region from the suspected feature region according to the output data change amount.

[0049] In this step, the key feature region represents the spatial range or region including the key feature. The feature represents the attribute or pattern that has an impact on the model performance. The feature includes the key feature, and the key feature represents the specific attribute or pattern that plays an important role in the model performance. It can be understood that the key feature region is a specific manifestation of the key feature. By identifying and locating the key feature region, the key feature can be better utilized.

[0050] By using the model analysis method for deep learning models provided by the embodiment of the present disclosure, through the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object, it is possible to obtain the area that contributes more to the activation of important neurons and the feature area that contributes more to the activation of multiple important neurons at the same time, that is, the suspected feature area. Since the synergy of multiple neurons in the deep learning model jointly determines the model output result, if a feature area contributes more to the activation of multiple important neurons at the same time, then the feature area is very likely to have a greater impact on the model output. The embodiment of the present disclosure first obtains the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object to determine the suspected feature area of ​​the classification object according to the mapping relationship. Then, the deep learning model is trained based on the classification object to obtain the output data change amount, and the key feature area is determined from the suspected feature area according to the output data change amount, so as to achieve accurate positioning of the key feature area. In the process of adaptive pruning of the deep learning model, accurately determining the key feature area helps to determine which part of the classification object is crucial to the model performance, making the pruning operation more targeted, and reducing the scale and computational complexity of the model while ensuring the model performance.

[0051] As an example, when the classification object is an image, the key feature is texture, color or shape, and the key feature area is a specific object including the above key feature, a part of a specific object including the above key feature, or a specific area including the above key feature. As another example, when the classification object is a text, the key feature is a specific vocabulary or grammatical structure, and the key feature area is a sentence including the above key feature or a paragraph including the above key feature. In a specific embodiment, the classification object is an image containing flowers, the key feature is the color of the petals, and the key feature area is the area in the classification object that presents the above key feature. For example, a certain type of flower has pink petals and the pink petals are located in the center of the flower. This color attribute is an important basis for distinguishing different types of flowers, and the area where the center of the flower is located is the key feature area. In another specific embodiment, the classification object is an image containing flowers, the key feature is the shape of the petals, and the key feature area is the area in the classification object that presents the above key feature. For example, a certain type of flower has slender oval petals and the oval petals are located in the left part of the flower, and the area where the left part of the flower is located is the key feature area. It can be seen from the above embodiments that key features are abstract attribute descriptions, while key feature regions are the embodiment of these attributes in a specific image space. By determining key feature regions, it is helpful to better understand key features, so as to use key features to optimize deep learning models, such as retaining neurons related to these key feature regions in adaptive pruning to ensure that the model maintains good performance after pruning.

[0052] The model analysis method for deep learning models provided by the embodiments of the present disclosure can determine the feature areas that are critical to model performance, namely the key feature areas, from the perspective of input data. When the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object is known, the neurons related to the key feature area can be known, which provides a reliable basis for weight evaluation in the pruning process. Compared with the related art, the embodiments of the present disclosure provide a new method for evaluating the importance of weights to determine the key weights, i.e., the weights of the neuron connections related to the key feature areas, including those weights that have small values ​​but play an important connection role in the model. Therefore, during the pruning process, the decrease in model accuracy caused by the mistaken deletion of key weights can be avoided, and the model can be ensured to maintain good performance after pruning while reducing the scale and computational complexity of the model.

[0053] Combination Figure 3 As shown, the embodiment of the present disclosure provides another model analysis method for a deep learning model, including:

[0054] S301, the processor determines the target neuron in the deep learning model.

[0055] In the deep learning model, multiple neurons are included, and different neurons are responsible for extracting and processing different feature information. In this embodiment, the target neuron, that is, the neuron to be studied, is first determined from the deep learning model. The target neuron can be a neuron in the deep learning model or multiple neurons that are related to each other.

[0056] S302: The processor configures the activation value of the target neuron to be 1 and configures the activation values ​​of other neurons to be 0, to obtain a fine-tuned deep learning model.

[0057] In this step, the activation value mask can be set to configure the activation value of the target neuron to 1, and the activation values ​​of all other neurons to 0. This ensures that only the target neuron is activated during subsequent data propagation, so that the relationship between the target neuron and the feature area of ​​the classification object can be observed separately.

[0058] S303: The processor obtains a mapping relationship between a neuron activation value of the deep learning model and a feature area of ​​the classification object based on fine-tuning the deep learning model.

[0059] In this step, after configuring the activation value of the target neuron, the deep learning model is fine-tuned. Fine-tuning can include adjusting the learning rate, optimizer parameters, etc. to ensure that the deep learning model can quickly adapt to the new activation value configuration and output stable results. Therefore, under the premise of ensuring the overall stability of the deep learning model, based on fine-tuning the deep learning model, the response between the target neuron and the characteristic area of ​​the classification object is captured, and the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object is obtained.

[0060] S304: The processor determines a suspected feature region of the classification object according to the mapping relationship.

[0061] In this step, the mapping relationship refers to the mapping relationship between the neuron activation value of the deep learning model obtained in step S303 and the feature region of the classification object. The suspected feature region represents a feature region that has a high contribution to the activation of multiple important neurons of the deep learning model at the same time.

[0062] S305: The processor performs model training on the deep learning model based on the classification object to obtain a change in output data.

[0063] S306: The processor determines a key feature region from the suspected feature region according to the output data change amount.

[0064] The model analysis method for a deep learning model provided by the embodiment of the present disclosure configures the activation value of the target neuron and fine-tunes the model so as to more accurately obtain the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object, thereby improving the accuracy and reliability of subsequent feature analysis.

[0065] Optionally, based on the fine-tuning of the deep learning model, a mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object is obtained, including: performing back propagation based on the fine-tuning of the deep learning model, back propagating from the target neuron to the input layer, and obtaining reference input data with the same dimension as the input layer; visualizing the reference input data to obtain the characteristic area of ​​the classification object corresponding to the target neuron activation value; replacing the target neuron to traverse each neuron in the deep learning model, and obtaining the characteristic area of ​​the classification object corresponding to each neuron activation value as the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object.

[0066] In this embodiment, on the fine-tuned deep learning model, starting from the target neuron, the gradient information is back-propagated from the target neuron to the input layer through the back-propagation algorithm to find the input data that can maximize the activation value of the target neuron, i.e., the reference input data. When the reference input data passes through the model, it can maximize the activation value of the target neuron, reflecting the sensitivity and preference of the target neuron to the input data. Then the reference input data is visualized, such as using a visualization tool (such as an activation heat map) to display, and more intuitively present the characteristic area of ​​the classification object corresponding to the target neuron activation value, so as to more intuitively understand the role of the target neuron in the model. In order to obtain the mapping relationship between the activation value of each neuron and the characteristic area of ​​the classification object in the deep learning model, traverse all neurons in the model and repeat the above steps. Each time it is traversed, the current neuron is used as the target neuron for back-propagation and visualization. After traversing all neurons in the model, the characteristic area of ​​the classification object corresponding to each neuron activation value is obtained, thereby constituting the mapping relationship between the neuron activation value of the deep learning model and the characteristic area of ​​the classification object.

[0067] In this embodiment, by traversing all neurons in the model, the mapping relationship between each neuron activation value and the characteristic area of ​​the classification object is obtained, so as to more comprehensively understand the working principle and feature extraction capability of the model. At the same time, by visually displaying the reference input data, the sensitivity and preference of each neuron to the input data can be more intuitively understood, thereby enhancing the interpretability of the model, so as to better understand the behavior and performance of the model.

[0068] Combination Figure 4 As shown, the embodiment of the present disclosure provides another model analysis method for a deep learning model, including:

[0069] S401, the processor obtains a mapping relationship between a neuron activation value of a deep learning model and a feature area of ​​a classification object.

[0070] S402: The processor determines a suspected feature region of the classification object according to the mapping relationship.

[0071] In this step, the mapping relationship refers to the mapping relationship between the neuron activation value of the deep learning model obtained in step S401 and the feature region of the classification object. The suspected feature region represents a feature region that has a high contribution to the activation of multiple important neurons of the deep learning model at the same time.

[0072] S403: The processor inputs the classification object into the deep learning model for model training to obtain original output data.

[0073] S404: The processor inputs the occlusion classification object into the deep learning model for model training to obtain reference output data.

[0074] The occluded classification object is obtained by occluding the local area of ​​the classification object with a sliding window.

[0075] S405: The processor obtains the output data variation according to the original output data and the reference output data.

[0076] S406: The processor determines a key feature region from the suspected feature region according to the output data variation.

[0077] In the model analysis method for a deep learning model provided by the embodiment of the present disclosure, a deep learning model is trained based on an occlusion algorithm to obtain a change in output data. The occlusion algorithm can simulate the situation where some information in the input data is missing and reveal the degree of dependence of the model on the missing information. By occluding different areas of the input data and observing the change in the model output data, the influence of different areas on the model performance can be further evaluated. Specifically, a local area of ​​the classification object is occluded by a sliding window technology to generate an occlusion classification object. The occlusion position characterizes the location of the occluded local area. The generated occlusion classification object is input into the deep learning model for training to obtain reference output data. By comparing the original output data with the reference output data, the influence of the occlusion position on the model output result is analyzed. According to the original output data and the reference output data, the change in output data is calculated to quantify the influence of the occlusion position on the model performance. By quantifying the change in output data, the performance of the model under different occlusion positions can be understood to identify which areas are occluded and may cause model recognition errors, and which areas are occluded and have no effect on the model recognition results, so as to determine the key feature areas from the suspected feature areas.

[0078] Optionally, the occlusion classification object is obtained in the following manner: the window size and occlusion position of the sliding window are determined; the occlusion position represents the location of the occluded local area in the occlusion classification object; based on the window size and the occlusion position, the area to be occluded of the classification object is determined; the pixel value of the area to be occluded is configured as a pixel threshold to obtain the occlusion classification object.

[0079] The window size determines how much area of ​​the classified object will be covered by each occlusion operation. In this embodiment, the window size of the sliding window needs to be determined by the technician according to the specific task requirements, computing resources and experimental results, and this application does not limit it. The occlusion position represents the location of the occluded local area in the occlusion classified object, that is, the position of the sliding window on the classified object.

[0080] In this embodiment, after the window size and the occlusion position are determined, it is possible to calculate which areas on the classified object will be occluded, i.e., the area to be occluded. The area to be occluded can be understood as the pixel area covered by the sliding window at the current position. After the area to be occluded is determined, the pixel value of the area to be occluded is set to a specific pixel threshold, such as 0, to ensure that the occlusion effect is obvious, thereby realizing the generation of the occluded classified object.

[0081] Optionally, the occlusion classification object is input into the deep learning model for model training to obtain reference output data, including: controlling the sliding window to move sequentially within the classification object according to the set sliding direction and step threshold to obtain multiple occlusion classification objects; and inputting multiple occlusion classification objects into the deep learning model for model training respectively to obtain parameter output data corresponding to each of the multiple occlusion classification objects.

[0082] In this embodiment, the sliding direction can be horizontal, vertical or oblique, etc. The step threshold determines the distance the window moves each time. In this embodiment, the step threshold can be equal to the width or length of the window size of the sliding window. For the position of each sliding window, the classification object area covered by it is occluded (for example, the pixel value of the area to be occluded is set to a specific threshold) to generate a corresponding occluded classification object. According to the set sliding direction and step threshold, the sliding window is controlled to move sequentially in the classification object to obtain a series of occluded classification objects with different occlusion positions. The generated multiple occlusion classification objects are respectively input into the deep learning model for training. For each occlusion classification object, the deep learning model will output corresponding parameter output data to reflect the classification performance and feature extraction ability of the model under different occlusion conditions.

[0083] Optionally, based on the output data change, the key feature area is determined from the suspected feature area, including: obtaining the output data change of each reference output data and the original output data; taking the occluded position of the sliding window corresponding to the reference output data with the largest output data change as the target position; and taking the suspected feature area containing the target position as the key feature area.

[0084] In this embodiment, the Euclidean distance method or the cosine similarity method can be used to calculate the data change between the reference output data and the original output data. The data change reflects the degree of influence of the occlusion position on the model output result. Then, the occlusion position of the sliding window corresponding to the reference output data with the largest data change is used as the target position, so that the position that has the greatest impact on the model output result and contains the key feature information of the classification object is used as the target position. After determining the target position, the suspected feature area containing the target position is used as the key feature area to achieve the identification of the key feature area. In this embodiment, the data change amount and the suspected feature area are combined to more accurately determine the key feature area.

[0085] Combination Figure 5 As shown, the embodiment of the present disclosure provides another model analysis method for a deep learning model, including:

[0086] S501, the processor obtains a mapping relationship between a neuron activation value of a deep learning model and a feature area of ​​a classification object.

[0087] S502: The processor determines a suspected feature region of the classification object according to the mapping relationship.

[0088] In this step, the mapping relationship refers to the mapping relationship between the neuron activation value of the deep learning model obtained in step S501 and the feature region of the classification object. The suspected feature region represents a feature region that has a high contribution to the activation of multiple important neurons of the deep learning model at the same time.

[0089] S503: The processor performs model training on the deep learning model based on the classification object to obtain a change in output data.

[0090] S504: The processor determines a key feature region from the suspected feature region according to the output data variation.

[0091] S505: The processor prunes the deep learning model according to the key feature area.

[0092] S506: The processor performs model training on the pruned deep learning model to obtain a target deep learning model.

[0093] The model analysis method for a deep learning model provided in the embodiment of the present disclosure determines the key feature area from the suspected feature area to ensure that the model can focus on the most important feature information. Then, according to the determined key feature area, the deep learning model is pruned to remove unimportant connections or parameters in the model to reduce the volume and computational complexity of the model while maintaining the performance of the model as much as possible. After the key feature area is determined, pruning can be performed more specifically to remove parts that are irrelevant to the key features or have a small contribution. The pruned deep learning model is retrained to restore the model performance that may be lost due to pruning, and to ensure that the model can still accurately identify the classified object while maintaining a small volume and low computational complexity. In the embodiment of the present disclosure, the unimportant parts of the model are removed by pruning, and the volume and computational complexity of the model are significantly reduced, thereby improving the operating efficiency and response speed of the model. Since it is based on the key feature area, the part that contributes more to the model performance can be retained as much as possible, thereby ensuring that the pruned model can still maintain a high recognition accuracy.

[0094] Optionally, the deep learning model is pruned according to the key feature area, including: using an objective function to calculate a first contribution of each neuron to the deep learning model; calculating a second contribution of each neuron to the deep learning model according to a gradient value of a loss function with respect to an activation value of each neuron; and pruning the deep learning model according to the first contribution, the second contribution and the key feature area.

[0095] In this embodiment, the objective function includes information gain and / or mutual information. The first contribution of each neuron to the deep learning model is calculated by using the objective function to quantify the contribution of each neuron to the improvement of model performance, so as to screen out neurons that have a greater impact on model performance. The gradient value reflects the sensitivity of the loss function to the neuron activation value, that is, the degree of influence of the change in the activation value of a certain neuron on the loss function. By calculating the gradient value, the role of each neuron in the model training process can be understood, so as to more accurately evaluate the importance of each neuron. After obtaining the first contribution and the second contribution of each neuron, the deep learning model is pruned in combination with the key feature area to more accurately evaluate the importance of each neuron to the deep learning model, avoid the misleading that may be caused by a single indicator, and improve the accuracy of pruning.

[0096] In some embodiments, the objective function includes information gain. Using the objective function to calculate the first contribution of each neuron to the deep learning model includes: considering the output of the i-th layer of neurons as feature Fi, calculating the information gain between feature Fi and the output variable Y of the deep learning model as the first contribution of the i-th layer of neurons to the deep learning model. The information gain calculation formula between feature Fi and the output variable Y of the deep learning model is as follows: IG(Fi, Y) = H(Y)-H(Y|Fi), where H(Y) represents the information entropy of the output variable Y of the deep learning model, and H(Y|Fi) represents the conditional entropy of the output variable Y after knowing the feature Fi. The information gain IG represents how much the uncertainty about the output variable Y can be reduced after knowing the feature Fi.

[0097] In some embodiments, the objective function includes mutual information. Using the objective function to calculate the first contribution of each neuron to the deep learning model includes: using the following formula to calculate the mutual information between the feature Fi output by the i-th layer of neurons and the output variable Y of the deep learning model: mutual information I(Fi, Y) = H(Fi) + H(Y) - H(Fi, Y); wherein, H(Y) represents the information entropy of the output variable Y of the deep learning model, H(Y) measures the uncertainty of the output variable Y, H(Fi) represents the information entropy of the feature Fi output by the i-th layer of neurons in the deep learning model, H(Fi) measures the uncertainty of the feature Fi, H(Fi, Y) represents the expected information contained in the input data (i.e., the feature Fi and the output variable Y) of the deep learning model under the joint distribution, and H(Fi, Y) measures the common uncertainty of the feature Fi and the output variable Y. Mutual information I(Fi, Y) measures the amount of shared information between the feature Fi and the output variable Y, that is, how much the uncertainty about the output variable Y can be reduced after knowing the feature Fi.

[0098] Gradient descent is a common optimization strategy in the training process of deep learning models. In each iteration, the gradient of the loss function with respect to the model parameters is calculated, and then the parameters are updated in the opposite direction of the gradient to reduce the loss. After the parameters are updated multiple times, the model's prediction ability will gradually improve. Therefore, gradient information can be used to evaluate the impact of neurons on the output results of the model. If a neuron often causes large gradient changes during training, it means that this neuron may have an important impact on the model as a whole. Therefore, in some embodiments, the second contribution of each neuron to the deep learning model is calculated based on the gradient value of the loss function with respect to the activation value of each neuron, including: obtaining the gradient change caused by each neuron in each iteration of the deep learning model; integrating the gradient change to obtain the second contribution of each neuron to the deep learning model. Specifically, the second contribution Where w represents the model parameters and L represents the loss function. It represents the gradient change caused by the i-th neuron with respect to the loss function during the iteration of the deep learning model.

[0099] Optionally, the deep learning model is pruned according to the first contribution, the second contribution and the key feature area, including: pruning neurons whose first contribution is less than a first threshold, whose second contribution is less than a second threshold and are not related to the key feature area. In this embodiment, the first contribution, the second contribution and the key feature area are comprehensively considered to avoid erroneous pruning that may be caused by a single indicator and improve the accuracy of pruning.

[0100] Combination Figure 6As shown, the embodiment of the present disclosure provides a feature analysis device 60 for a deep learning model, including a processor (processor) 600 and a memory (memory) 601. Optionally, the device 60 may also include a communication interface (CommunicationInterface) 602 and a bus 603. Among them, the processor 600, the communication interface 602, and the memory 601 can communicate with each other through the bus 603. The communication interface 602 can be used for information transmission. The processor 600 can call the logic instructions in the memory 601 to execute the feature analysis method for the deep learning model of the above embodiment.

[0101] In addition, the logic instructions in the memory 601 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0102] The memory 601 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 600 executes the function application and data processing by running the program instructions / modules stored in the memory 601, that is, the feature analysis method for the deep learning model in the above embodiment is implemented.

[0103] The memory 601 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 601 may include a high-speed random access memory and may also include a non-volatile memory.

[0104] An embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein the program executes the above-mentioned feature analysis method for a deep learning model when it is run.

[0105] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, such as: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0106] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0107] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0108] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0109] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A feature analysis method for a deep learning model, characterized in that: include: Obtain the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object; Determining a suspected feature region of the classification object according to the mapping relationship; Suspected feature regions represent feature regions that simultaneously contribute highly to the activation of multiple important neurons in the deep learning model; Perform model training on the deep learning model based on the classification object to obtain the output data change; According to the output data variation, key feature areas are determined from the suspected feature areas.

2. The feature analysis method according to claim 1, characterized in that: Obtain the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object, including: Identify target neurons in deep learning models; Configure the activation value of the target neuron to 1 and the activation values ​​of other neurons to 0 to obtain a fine-tuned deep learning model; Based on the fine-tuning of the deep learning model, the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object is obtained.

3. The feature analysis method according to claim 2, characterized in that: Based on the fine-tuning of the deep learning model, the mapping relationship between the neuron activation value of the deep learning model and the feature area of ​​the classification object is obtained, including: Perform back propagation based on the fine-tuned deep learning model, back propagating from the target neuron to the input layer to obtain reference input data with the same dimension as the input layer; Visualize the reference input data to obtain the characteristic region of the classification object corresponding to the target neuron activation value; The target neuron is replaced to traverse each neuron in the deep learning model, and the characteristic region of the classification object corresponding to each neuron activation value is obtained as a mapping relationship between the neuron activation value of the deep learning model and the characteristic region of the classification object.

4. The feature analysis method according to any one of claims 1 to 3, characterized in that: The deep learning model is trained based on the classification object to obtain the output data change, including: Input the classification object into the deep learning model for model training to obtain the original output data; Input the occluded classification object into the deep learning model for model training to obtain reference output data; the occluded classification object is obtained by occluding the local area of ​​the classification object with a sliding window; The output data variation is obtained according to the original output data and the reference output data.

5. The feature analysis method according to claim 4, characterized in that: Obtain the occlusion classification object as follows: Determine the window size and occlusion position of the sliding window; the occlusion position represents the location of the occluded local area in the occlusion classification object; Determine the to-be-occluded area of ​​the classified object according to the window size and the occlusion position; Configure the pixel value of the area to be occluded as the pixel threshold to obtain the occlusion classification object.

6. The feature analysis method according to claim 5, characterized in that: Input the occlusion classification object into the deep learning model for model training to obtain reference output data, including: According to the set sliding direction and step size threshold, the sliding window is controlled to move sequentially in the classification object to obtain multiple occluded classification objects; Multiple occlusion classification objects are respectively input into the deep learning model for model training to obtain parameter output data corresponding to each of the multiple occlusion classification objects.

7. The feature analysis method according to claim 6, characterized in that: According to the output data change, key feature areas are determined from the suspected feature areas, including: Obtaining an output data variation between each reference output data and the original output data; The occluded position of the sliding window corresponding to the reference output data with the largest output data change is taken as the target position; The suspected feature area containing the target position is taken as the key feature area.

8. The feature analysis method according to any one of claims 1 to 3, characterized in that: Also includes: Prune the deep learning model based on key feature areas; Perform model training on the pruned deep learning model to obtain the target deep learning model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program, when executed, executes the feature analysis method for a deep learning model according to any one of claims 1 to 8.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the feature analysis method for a deep learning model according to any one of claims 1 to 8 through the computer program.

Images