A method, device and storage medium for locating abnormalities in a deep learning framework
By layering the first neural network model of the deep learning framework into multiple sub-models and running on the reference framework and test framework to calculate the signal-to-noise ratio, the problem of poor abnormal positioning performance in the prior art is solved, achieving more efficient positioning accuracy.
Patent Information
- Application Number
- CN202010937170.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-09-08
AI Technical Summary
The existing deep learning framework exception positioning methods require a long processing time and processing resources, resulting in poor performance.
By layering the first neural network model into multiple submodels and running these submodels separately on the reference frame and the test frame, the signal-to-noise ratio of each submodel is calculated to locate the abnormal location.
Improve the accuracy and efficiency of abnormal positioning, and reduce processing time and resource consumption.
Smart Images

Figure CN112070235B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of deep learning technology, and in particular to an anomaly locating method, device and storage medium for a deep learning framework. Background Art
[0002] More and more frameworks for programming and implementing machine learning algorithms are widely used. Such systems are generally deployed on various servers, PCs, and web pages and support high-performance numerical calculations of graphics processing units (GPUs). They are widely used in product development and scientific research in various fields.
[0003] The method of detecting abnormal positions of such systems is generally to test the internal units of the system to find the differences between the internal units and the design targets. However, this detection method requires a relatively long processing time and uses more processing resources.
[0004] Therefore, improving the performance of locating abnormal positions of the framework is a technical problem that needs to be solved. Summary of the invention
[0005] In order to overcome the problems existing in the related art, the present invention provides a method, device and storage medium for locating anomalies in a deep learning framework.
[0006] According to a first aspect of an embodiment of the present disclosure, a method for locating anomalies in a deep learning framework is provided, comprising:
[0007] Determine the total number of layers of the first neural network model;
[0008] Determining a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers;
[0009] Using the same input data, the plurality of sub-models are sequentially run on a reference frame and a test frame respectively, and a signal-to-noise ratio corresponding to each sub-model is determined according to the difference between the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model;
[0010] The abnormal position of the test frame is located according to the signal-to-noise ratios corresponding to all sub-models.
[0011] In one embodiment, determining the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame includes:
[0012] The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
[0013] In one implementation, locating the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models includes:
[0014] Determine a plurality of model pairs, each model pair includes two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model;
[0015] Calculate the reduction in the signal-to-noise ratio of the second submodel compared to the noise ratio of the first submodel in each model pair;
[0016] Determine each model pair that satisfies the reduction range greater than the set range;
[0017] Select the model pair with the lowest total number of layers in the first sub-model from among the model pairs that satisfy the reduction amplitude greater than the set amplitude, or select the model pair with the lowest total number of layers in the second sub-model;
[0018] It is determined that the last layer of the second sub-model in the selected model pair is the target layer, and it is determined that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
[0019] In one embodiment, the method further comprises: determining a second neural network model; the nodes of the first layer of the second neural network model are nodes of non-numerical transformation type;
[0020] The locating the abnormal position of the test framework according to the signal-to-noise ratios corresponding to all sub-models includes:
[0021] When the signal-to-noise ratio corresponding to the first submodel of the ordered multiple submodels is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, the first layer of the second neural network is run on the reference frame and the test frame respectively using the same input data, and the signal-to-noise ratio corresponding to the first layer of the second neural network is determined according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame; when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, it is determined that the abnormal position of the test frame is a frame structure.
[0022] In one embodiment, the method further includes: when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value, determining that the abnormal position of the test framework is the operator corresponding to the first layer of the first neural network model in the test framework.
[0023] According to a second aspect of an embodiment of the present disclosure, a device for locating anomalies in a deep learning framework is provided, comprising:
[0024] A first determination module is configured to determine the total number of layers of the first neural network model;
[0025] a second determination module configured to determine a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers;
[0026] A running module configured to sequentially run the plurality of sub-models on a reference frame and a test frame respectively using the same input data;
[0027] A third determination module is configured to determine the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model;
[0028] The positioning module is configured to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models.
[0029] In one embodiment, the third determination module is further configured to determine the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame using the following method:
[0030] The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
[0031] In one embodiment, the positioning module is further configured to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models using the following method, including:
[0032] Determine a plurality of model pairs, each model pair includes two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model;
[0033] Calculate the reduction in the signal-to-noise ratio of the second submodel compared to the noise ratio of the first submodel in each model pair;
[0034] Determine each model pair that satisfies the reduction range greater than the set range;
[0035] Select the model pair with the lowest total number of layers in the first sub-model from among the model pairs that satisfy the reduction amplitude greater than the set amplitude, or select the model pair with the lowest total number of layers in the second sub-model;
[0036] It is determined that the last layer of the second sub-model in the selected model pair is the target layer, and it is determined that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
[0037] In one embodiment, the device further comprises:
[0038] A fourth determination module is configured to determine a second neural network model; the nodes of the first layer of the second neural network model are nodes of a non-numerical transformation type;
[0039] The positioning module is further configured to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models using the following method:
[0040] When the signal-to-noise ratio corresponding to the first submodel of the ordered multiple submodels is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, the first layer of the second neural network is run on the reference frame and the test frame respectively using the same input data, and the signal-to-noise ratio corresponding to the first layer of the second neural network is determined according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame; when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, it is determined that the abnormal position of the test frame is a frame structure.
[0041] In one embodiment, the positioning module is also configured to determine that the abnormal position of the test frame is the operator corresponding to the first layer of the first neural network model in the test frame when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value.
[0042] According to a third aspect of an embodiment of the present disclosure, a device for locating anomalies in a deep learning framework is provided, comprising:
[0043] processor;
[0044] a memory for storing processor-executable instructions;
[0045] Wherein, the processor is configured to execute the method.
[0046] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, which enables the mobile terminal to execute the method when instructions in the storage medium are executed by a processor of the mobile terminal.
[0047] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by setting a sub-model including a plurality of continuous layers starting from the first layer in the first neural network model, the structure of the first neural network model is layered, and multiple sub-models are used for testing, and the abnormal position of the test frame is located according to the signal-to-noise ratio obtained after the multiple sub-models are respectively run on the reference frame and the test frame, and different sub-models are used to simulate the effects of different processing stages in the first neural network model, thereby improving the positioning accuracy.
[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0050] Figure 1 is a flow chart of a method for locating anomalies in a deep learning framework according to an exemplary embodiment;
[0051] Figure 2 is a structural diagram of a device for locating anomalies in a deep learning framework according to an exemplary embodiment;
[0052] Figure 3 It is a structural diagram of a device for locating anomalies in a deep learning framework according to an exemplary embodiment. DETAILED DESCRIPTION
[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0054] The present disclosure provides a method for locating anomalies in a deep learning framework. Figure 1 , Figure 1 FIG. 1 is a flowchart of a method for locating anomalies in a deep learning framework according to an exemplary embodiment. Figure 1 As shown, this method includes:
[0055] Step S11, determining the total number of layers of the first neural network model.
[0056] Step S12: determining a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers.
[0057] Step S13, using the same input data to sequentially run the multiple sub-models on the reference frame and the test frame, respectively, and determining the signal-to-noise ratio corresponding to each sub-model based on the difference in the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model.
[0058] Step S14: locating abnormal positions of the test frame according to the signal-to-noise ratios corresponding to all sub-models.
[0059] The reference framework and the test framework are both frameworks used for programming and implementing machine learning algorithms. For example, the reference framework is TensorFlow and the test framework is MACE.
[0060] For example, the first neural network model includes 5 layers, that is, the total number of layers is 5. Five sub-models are determined, and the 5 sub-models include the 1st layer, the 1st to 2nd layers, the 1st to 3rd layers, the 1st to 4th layers, and the 1st to 5th layers of the first neural network model respectively.
[0061] In this embodiment, the structure of the first neural network model is layered by setting a sub-model including multiple continuous layers starting from the first layer in the first neural network model, and multiple sub-models are used for testing. The abnormal position of the test frame is located according to the signal-to-noise ratio obtained after the multiple sub-models are respectively run on the reference frame and the test frame, and different sub-models are used to simulate the effects of different processing stages in the first neural network model to improve the positioning accuracy.
[0062] The present disclosure provides a method for locating anomalies in a deep learning framework, the method comprising: Figure 1 The method shown in the figure, and: in step S13, determining the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame, comprising:
[0063] The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
[0064] Among them, the difference between the output result of each sub-model in the reference frame and the output result of the test frame is the difference between the output result of this sub-model in the reference frame and the output result of the test frame, or the difference between the output result of this sub-model in the test frame and the output result of the reference frame, or the absolute value of the difference between the two.
[0065] The present disclosure provides a method for locating anomalies in a deep learning framework, the method comprising: Figure 1 The method shown, and: in step S14, locating the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models, including:
[0066] Step S141, determining a plurality of model pairs, each model pair including two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model;
[0067] Step S142, calculating the reduction of the signal-to-noise ratio of the second sub-model in each model pair compared with the noise ratio of the first sub-model;
[0068] Step S143, determining each model pair that satisfies the requirement that the reduction amplitude is greater than the set amplitude;
[0069] Step S144, selecting the model pair with the lowest total number of layers of the first sub-model from the model pairs that satisfy the reduction amplitude greater than the set amplitude, or selecting the model pair with the lowest total number of layers of the second sub-model;
[0070] Step S145, determining that the last layer of the second sub-model in the selected model pair is the target layer, and determining that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
[0071] For example: set the amplitude to 0.5. The first neural network model includes 5 layers, and the 5 sub-models are model A, model B, model C, model D and model E. Model A includes the first layer of the first neural network model, model B includes the first to second layers of the first neural network model, model C includes the first to third layers of the first neural network model, model D includes the first to fourth layers of the first neural network model, and model E includes the first to fifth layers of the first neural network model.
[0072] The signal-to-noise ratios of the five sub-models are 102, 100, 49, 48, and 20, respectively.
[0073] Four model pairs are determined, each model pair includes two sub-models, and the total number of layers of the first sub-model is less than the total number of layers of the second sub-model. That is, each model pair includes two adjacent sub-models. The first model pair includes model A and model B, the second model pair includes model B and model C, the third model pair includes model C and model D, and the fourth model pair includes model D and model E.
[0074] Calculate the reduction of the signal-to-noise ratio of the second sub-model in each model pair compared to the noise ratio of the first sub-model. The corresponding reductions for the four model pairs are: 0.0196, 0.51, 0.02, and 0.58.
[0075] It is determined that the model pairs that satisfy the reduction amplitude greater than the set amplitude are the second model pair and the fourth model pair.
[0076] The second model pair is selected from the second model pair and the fourth model pair, wherein the model pair with the lowest total number of layers of the first sub-model or the model pair with the lowest total number of layers of the second sub-model is selected, and the second model pair is selected. The second model pair includes model B and model C.
[0077] It is determined that the last layer of the second sub-model in the second model pair, namely model C, is the target layer, and this target layer is the third layer in the first neural network model. It is determined that the abnormal position of the test framework is the operator corresponding to this target layer in the test framework.
[0078] The present disclosure provides a method for locating anomalies in a deep learning framework, the method comprising: Figure 1 The method shown in the figure also includes: determining a second neural network model; the nodes of the first layer of the second neural network model are nodes of non-numerical transformation type;
[0079] In step S14, the abnormal position of the test frame is located according to the signal-to-noise ratios corresponding to all sub-models, including:
[0080] When the signal-to-noise ratio corresponding to the first submodel of the ordered multiple submodels is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, the first layer of the second neural network is run on the reference frame and the test frame respectively using the same input data, and the signal-to-noise ratio corresponding to the first layer of the second neural network is determined according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame; when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, it is determined that the abnormal position of the test frame is a frame structure.
[0081] Among them, the operator corresponding to the first layer of the first neural network model in the reference frame is different from the operator corresponding to the first layer of the second neural network model in the reference frame. The operator corresponding to the first layer of the first neural network model in the reference frame is a node of the numerical transformation class. The operator corresponding to the first layer of the second neural network model in the reference frame is a node of the non-numerical transformation class.
[0082] The operator corresponding to the first layer of the first neural network model in the test framework is different from the operator corresponding to the first layer of the second neural network model in the test framework. The operator corresponding to the first layer of the first neural network model in the test framework is a node of the numerical transformation class. The operator corresponding to the first layer of the second neural network model in the test framework is a node of the non-numerical transformation class.
[0083] Among them, operators of the numerical transformation type are generally calculation operators, such as addition, subtraction, multiplication, division, multiple powers, convolution, etc.
[0084] Non-numeric transformation operators refer to operators that reformat the data without changing the content of the data, for example, converting a vector of 12 data arranged in sequence into a matrix arranged in a 3-by-4 format.
[0085] In one embodiment, when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value, it is determined that the abnormal position of the test framework is the operator corresponding to the first layer of the first neural network model in the test framework.
[0086] The present disclosure provides an abnormality location device for a deep learning framework. Figure 2 , Figure 2 FIG. 1 is a structural diagram of an abnormality location device of a deep learning framework according to an exemplary embodiment. Figure 2 As shown, the device comprises:
[0087] The first determination module 201 is configured to determine the total number of layers of the first neural network model.
[0088] The second determination module 202 is configured to determine a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers.
[0089] The running module 203 is configured to use the same input data to sequentially run the multiple sub-models on the reference frame and the test frame respectively.
[0090] The third determination module 204 is configured to determine the signal-to-noise ratio corresponding to each sub-model based on the difference in the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model.
[0091] The positioning module 205 is configured to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models.
[0092] The present disclosure provides an abnormality location device for a deep learning framework, which includes Figure 2 The device shown in the figure, and: the third determination module 204 is further configured to determine the signal-to-noise ratio corresponding to each sub-model according to the difference in the running results of each sub-model on the reference frame and the test frame using the following method:
[0093] The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
[0094] The present disclosure provides an abnormality location device for a deep learning framework, which includes Figure 2 The device shown in the figure, and: the positioning module 205 is further configured to use the following method to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models, including:
[0095] Determine a plurality of model pairs, each model pair includes two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model;
[0096] Calculate the reduction in the signal-to-noise ratio of the second submodel compared to the noise ratio of the first submodel in each model pair;
[0097] Determine each model pair that satisfies the reduction range greater than the set range;
[0098] Select the model pair with the lowest total number of layers in the first sub-model from among the model pairs that satisfy the reduction amplitude greater than the set amplitude, or select the model pair with the lowest total number of layers in the second sub-model;
[0099] It is determined that the last layer of the second sub-model in the selected model pair is the target layer, and it is determined that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
[0100] The present disclosure provides an abnormality location device for a deep learning framework, which includes Figure 2The device shown also includes: a fourth determination module, configured to determine a second neural network model; the operators corresponding to the first layer of the second neural network model in the reference frame and the test frame are the same as the operators corresponding to the first layer of the first neural network model in the reference frame and the test frame; the nodes of the first layer of the second neural network model are nodes of non-numerical transformation class.
[0101] The positioning module 205 is further configured to use the following method to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models:
[0102] When the signal-to-noise ratio corresponding to the first submodel of the ordered multiple submodels is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, the first layer of the second neural network is run on the reference frame and the test frame respectively using the same input data, and the signal-to-noise ratio corresponding to the first layer of the second neural network is determined according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame; when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, it is determined that the abnormal position of the test frame is a frame structure.
[0103] In one embodiment, the positioning module 205 is also configured to determine that the abnormal position of the test framework is the operator corresponding to the first layer of the first neural network model in the test framework when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value.
[0104] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0105] Figure 3 1 is a block diagram of an abnormality location device 300 of a deep learning framework according to an exemplary embodiment. For example, the device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0106] Reference Figure 3 , the device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .
[0107] The processing component 302 generally controls the overall operation of the device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.
[0108] The memory 304 is configured to store various types of data to support operations on the device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0109] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.
[0110] The multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and / or a rear camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0111] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.
[0112] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0113] The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the device 300. For example, the sensor assembly 314 can detect the open / closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, the sensor assembly 314 can also detect the position change of the device 300 or a component of the device 300, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 300, and the temperature change of the device 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0114] The communication component 316 is configured to facilitate wired or wireless communication between the device 300 and other devices. The device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0115] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0116] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, and the instructions can be executed by the processor 320 of the device 300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0117] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0118] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for locating anomalies in a deep learning framework, characterized in that: include: Determine the total number of layers of the first neural network model; Determining a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers; Using the same input data, the plurality of sub-models are sequentially run on a reference frame and a test frame respectively, and a signal-to-noise ratio corresponding to each sub-model is determined according to the difference between the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model; Locating abnormal positions of the test framework according to the signal-to-noise ratios corresponding to all sub-models; Wherein, locating the abnormal position of the test framework according to the signal-to-noise ratios corresponding to all sub-models includes: Determine a plurality of model pairs, each model pair includes two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model; Calculate the reduction in the signal-to-noise ratio of the second submodel compared to the noise ratio of the first submodel in each model pair; Determine each model pair that satisfies the reduction range greater than the set range; Select the model pair with the lowest total number of layers in the first sub-model from among the model pairs that satisfy the reduction amplitude greater than the set amplitude, or select the model pair with the lowest total number of layers in the second sub-model; It is determined that the last layer of the second sub-model in the selected model pair is the target layer, and it is determined that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
2. The method according to claim 1, characterized in that The determining the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame includes: The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
3. The method according to claim 1, characterized in that The method further comprises: determining a second neural network model; the nodes of the first layer of the second neural network model are nodes of non-numerical transformation type; The method of locating the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models includes: when the signal-to-noise ratio corresponding to the first sub-model in the ordered multiple sub-models is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, using the same input data to run the first layer of the second neural network on a reference frame and a test frame respectively, determining the signal-to-noise ratio corresponding to the first layer of the second neural network according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame, and when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, determining that the abnormal position of the test frame is a frame structure.
4. The method according to claim 3, characterized in that The method also includes: when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value, determining that the abnormal position of the test framework is the operator corresponding to the first layer of the first neural network model in the test framework.
5. An abnormality location device for a deep learning framework, characterized in that: include: A first determination module is configured to determine the total number of layers of the first neural network model; a second determination module configured to determine a plurality of ordered sub-models according to the first neural network model, wherein the number of the plurality of sub-models is the same as the total number of layers, each sub-model includes a plurality of continuous layers starting from the first layer in the first neural network model, and different sub-models include different numbers of layers; A running module configured to sequentially run the plurality of sub-models on a reference frame and a test frame respectively using the same input data; A third determination module is configured to determine the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame, wherein the reference frame and the test frame both include operators corresponding to each layer of the first neural network model; A positioning module, configured to locate an abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models; The positioning module is further configured to locate the abnormal position of the test framework according to the signal-to-noise ratios corresponding to all sub-models using the following method, including: Determine a plurality of model pairs, each model pair includes two sub-models whose total number of layers differs by 1, and the total number of layers of the first sub-model in each model pair is less than the total number of layers of the second sub-model; Calculate the reduction in the signal-to-noise ratio of the second submodel compared to the noise ratio of the first submodel in each model pair; Determine each model pair that satisfies the reduction range greater than the set range; Select the model pair with the lowest total number of layers in the first sub-model from among the model pairs that satisfy the reduction amplitude greater than the set amplitude, or select the model pair with the lowest total number of layers in the second sub-model; It is determined that the last layer of the second sub-model in the selected model pair is the target layer, and it is determined that the abnormal position of the test framework is the operator corresponding to the target layer in the test framework.
6. The device according to claim 5, characterized in that The third determination module is further configured to determine the signal-to-noise ratio corresponding to each sub-model according to the difference between the running results of each sub-model on the reference frame and the test frame using the following method: The operating result of each sub-model in the reference frame is taken as a signal, the difference between the operating result of the sub-model in the reference frame and the operating result of the test frame is taken as noise, and the signal-to-noise ratio of the sub-model is calculated based on the signal and the noise.
7. The device according to claim 5, characterized in that The device also includes: A fourth determination module is configured to determine a second neural network model; the nodes of the first layer of the second neural network model are nodes of a non-numerical transformation type; The positioning module is further configured to locate the abnormal position of the test frame according to the signal-to-noise ratios corresponding to all sub-models using the following method: When the signal-to-noise ratio corresponding to the first submodel of the ordered multiple submodels is less than a set value and the nodes of the first layer of the first neural network model are nodes of the numerical transformation type, the first layer of the second neural network is run on the reference frame and the test frame respectively using the same input data, and the signal-to-noise ratio corresponding to the first layer of the second neural network is determined according to the difference between the running results of the first layer of the second neural network on the reference frame and the test frame; when the signal-to-noise ratio corresponding to the first layer of the second neural network is less than or equal to the set value, it is determined that the abnormal position of the test frame is a frame structure.
8. The device according to claim 7, characterized in that The positioning module is also configured to determine that the abnormal position of the test framework is the operator corresponding to the first layer of the first neural network model in the test framework when the signal-to-noise ratio corresponding to the first layer of the second neural network is greater than the set value.
9. A device for locating anomalies in a deep learning framework, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method of any one of claims 1 to 4.
10. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the method of any one of claims 1 to 4.
