A reasoning method, apparatus, device, and computer storage medium

Abstract

Embodiments of the present application disclose a reasoning method, device and equipment and a computer storage medium. The method comprises: inputting a to-be-detected image into a target reasoning model to obtain a prediction result; in a case where it is detected that the to-be-detected image is an abnormal image, updating original model features of the target reasoning model according to different proportions according to an abnormal category of the abnormal image; and in a case where it is detected that the to-be-detected image is a normal image, outputting the prediction result.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and in particular to a reasoning method, apparatus, device, and computer storage medium. Background Technology

[0002] With the rapid development of internet information technology, the business scenarios in which internet technology is applied are also gradually increasing. Due to the characteristics of internet applications, such as rapid scenario changes and short update cycles, model updates not only require high coverage and accuracy but also high update efficiency.

[0003] However, during the model update process, due to the significant differences between the sample images and the original training images, the accuracy of the model may decrease instead of increase when encountering abnormal samples when using conventional update strategies. Moreover, the participation of abnormal samples in the update will increase the model parameters and the required computing resources, which will bring difficulties to the model update work. Summary of the Invention

[0004] This application proposes a reasoning method, apparatus, device, and computer storage medium. The technical solution of this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a reasoning method, the method comprising:

[0006] The image to be detected is input into the target inference model to obtain the prediction result;

[0007] If the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image.

[0008] If the image to be detected is a normal image, the prediction result is output.

[0009] In some embodiments, the method further includes:

[0010] The difference between the image to be detected and the initial training image is calculated using the anomaly detection module.

[0011] If the difference value is greater than the abnormal threshold, then the detection result indicates that the image to be detected is an abnormal image, and the abnormality category of the abnormal image is determined.

[0012] If the difference value is less than or equal to the abnormal threshold, then the detection result indicates that the image to be detected is the normal image.

[0013] In some embodiments, determining the anomaly category of the abnormal image includes:

[0014] The abnormal image is measured and calculated to determine the difference value of the abnormal image;

[0015] The anomaly category of the anomaly image is determined based on the difference between the difference value of the anomaly image and the anomaly threshold; wherein the anomaly category includes at least one of the following: novel type sample, difficult-to-learn sample, or drift sample.

[0016] In some embodiments, when the image to be detected is detected to be an abnormal image, the method further includes:

[0017] The abnormal image is stored in the abnormal database;

[0018] When the number of abnormal images in the abnormal database reaches the update threshold, the target inference model is updated using the abnormal images in the abnormal database.

[0019] In some embodiments, when the number of abnormal images in the abnormal database reaches a preset update threshold, the method further includes:

[0020] Based on the data annotation of the abnormal image, an annotated abnormal image is generated;

[0021] Accordingly, updating the original model features of the target inference model according to different proportions based on the anomaly category of the abnormal image includes:

[0022] Statistical analysis was performed on abnormal head images belonging to new types of samples, abnormal images belonging to difficult-to-learn samples, and abnormal images belonging to drift samples, and the data offset value was determined based on the statistical results.

[0023] If the data offset value is greater than the offset threshold, the target inference model is updated using the labeled anomaly image based on the first type of update strategy; or...

[0024] If the data offset value is less than or equal to the offset threshold, the target inference model is updated using the labeled anomaly image based on the second type of update strategy.

[0025] Wherein, the proportion of the original model features of the target inference model updated in the first type of update strategy is less than the proportion of the original model features of the target prediction model updated in the second type of update strategy.

[0026] In some embodiments, the first type of update strategy includes at least a greedy clustering update strategy, and the target inference model includes at least a memory module;

[0027] The step of updating the target inference model using the labeled anomaly image based on the first type of update strategy includes:

[0028] The original model features to be updated in the target inference model are stored in the memory module;

[0029] Based on the greedy clustering update strategy, the memory module is updated using the labeled abnormal images in the abnormal database.

[0030] In some embodiments, the step of updating the memory module using the labeled anomaly images in the anomaly database based on the greedy clustering update strategy includes:

[0031] Based on the labeled abnormal images in the abnormal database, an incremental feature set is determined;

[0032] Obtain the current feature set of the memory module;

[0033] Perform an intersection operation on the incremental feature set and the current feature set to obtain the target intersection;

[0034] Clustering is performed on the target intersection to obtain a new feature set, and the memory module is updated with the parameters of the new feature set.

[0035] Secondly, embodiments of this application provide a reasoning device, including:

[0036] The inference module is configured to input the image to be detected into the target inference model to obtain the prediction result;

[0037] If the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image.

[0038] If the image to be detected is a normal image, the prediction result is output.

[0039] Thirdly, embodiments of this application provide an electronic device, including:

[0040] Memory is used to store computer programs that can run on a processor;

[0041] A processor for executing the method as described in any one of the first aspects when running the computer program.

[0042] Fourthly, embodiments of this application provide a computer storage medium storing a computer program that, when executed by at least one processor, implements the method described in any of the first aspects.

[0043] This application provides an inference method, apparatus, device, and computer storage medium. The method includes: inputting a target inference model with an image to be detected to obtain a prediction result; updating the original features of the target inference model at different proportions according to the anomalous category of the image if the image to be detected is an anomalous image; and outputting the prediction result if the image to be detected is a normal image. This method, by using anomalous images to partially update the original features of the target inference model, not only reduces computational resources and complexity but also improves the inference capability of the target inference model while maintaining its accuracy. Attached Figure Description

[0044] Figure 1 A flowchart illustrating a reasoning method provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating another reasoning method provided in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram illustrating the working process of an inference device provided in an embodiment of this application;

[0047] Figure 4 This is a schematic diagram of the composition structure of a reasoning device provided in an embodiment of this application;

[0048] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0050] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0052] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first, second, third" used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0053] In related technologies, with the rapid development of artificial intelligence, various edge inference models are constantly being implemented in actual production. For inference models, a large number of training samples are needed to determine various model parameters before they can be used in actual production processes. However, in actual edge model inference scenarios, the input real data samples may differ from the training set. At this time, the inference model cannot accurately judge them, resulting in unsatisfactory inference results. How to quickly update model parameters based on these data while satisfying the following conditions: (1) the model accuracy increases significantly on new data; (2) the model accuracy does not decrease significantly on the original data; (3) the model parameters (or knowledge base size) do not increase indefinitely; (4) the required computing resources do not increase with updates, etc., is a major challenge.

[0054] Based on this, embodiments of this application provide a reasoning method. The basic idea of ​​this method is as follows: input the image to be detected into the target reasoning model to obtain a prediction result; when the image to be detected is an abnormal image, update the original model features of the target reasoning model according to different proportions based on the abnormality category of the abnormal image; when the image to be detected is a normal image, output the prediction result. In this way, by using abnormal images to partially update the original features of the target reasoning model, not only are computational resources reduced and computational complexity lowered, but the reasoning ability of the target reasoning model can also be improved while ensuring the accuracy of the target reasoning model.

[0055] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0056] In one embodiment of this application, see Figure 1 It shows a flowchart of a reasoning method provided in an embodiment of this application, such as... Figure 1 As shown, the method may include:

[0057] S101: Input the image to be detected into the target inference model to obtain the prediction result.

[0058] It should be noted that the inference method provided in this application can be applied to various electronic devices with computing capabilities. Here, electronic devices can be such as computers, smartphones, tablets, laptops, handheld computers, personal digital assistants (PDAs), servers, etc. In this application embodiment, the electronic device can specifically be a mobile edge computing (MEC) server, but this is not specifically limited here.

[0059] In this embodiment, the inference model can be a model from various fields, such as image processing, speech recognition, natural language recognition, etc. Specifically, the inference model is used to perform specific detections on the image to be detected in order to output a prediction result.

[0060] S102: When the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image.

[0061] It should be noted that during the use of the inference model, the image to be detected needs to be judged to determine whether it is an abnormal image. If the image to be detected is an abnormal image, these abnormal images are used to update the target inference model.

[0062] It should also be noted that when abnormal images are detected, they can be stored centrally. When the number of abnormal images reaches a preset threshold, the centrally stored abnormal images can be used to update the target inference model.

[0063] In some embodiments, the method may further include the following steps in detecting abnormal images:

[0064] The difference between the image to be detected and the initial training image is calculated using the anomaly detection module.

[0065] If the difference value is greater than the abnormal threshold, then the detection result indicates that the image to be detected is an abnormal image, and the abnormality category of the abnormal image is determined.

[0066] If the difference value is less than or equal to the abnormal threshold, then the detection result indicates that the image to be detected is the normal image.

[0067] It should be noted that the difference value is calculated using at least one metric score. The higher the difference value, the greater the deviation between the anomalous image and the original training samples. Specifically, the difference value corresponding to the image to be detected can be obtained by interpolating at least one metric score using the anomaly detection module. Here, interpolation is not mandatory; methods such as mean calculation can also be used to determine the difference value.

[0068] In one specific embodiment, determining the anomaly category of the abnormal image may include:

[0069] The abnormal image is measured and calculated to determine the difference value of the abnormal image;

[0070] The anomaly category of the anomaly image is determined based on the difference between the difference value of the anomaly image and the anomaly threshold; wherein the anomaly category includes at least one of the following: novel type sample, difficult-to-learn sample, or drift sample.

[0071] It should be noted that, in this embodiment, based on the degree of deviation between the abnormal image and the original training samples, the abnormal images are divided into three abnormal categories: novel samples, difficult-to-learn samples, or drift samples. A difficult-to-learn sample refers to an abnormal image that falls within the coverage area of ​​the original training samples but is difficult to predict; a drift sample refers to an abnormal image that deviates from the original training samples, but the deviation is not significant; and a novel sample refers to an abnormal image that is of a different type from the original training samples.

[0072] For example, suppose the target prediction model is used to identify cats in the image to be detected. The original training samples are a large number of related pictures of orange cats. Then, the hard-learning samples, drift samples, and new type samples can be understood as follows: the hard-learning samples are still orange cat pictures, but they are different from the original training samples; the drift samples may be black cat pictures; and the new type samples may be dog pictures.

[0073] In one specific embodiment, the difference values ​​of hard-to-learn samples, drift samples, and novel type samples increase sequentially. Correspondingly, the anomaly threshold may include a first anomaly threshold and a second anomaly threshold, with the first anomaly threshold being less than the second anomaly threshold. If the difference value of an anomaly image is less than or equal to the first anomaly threshold, the anomaly image is determined to be a hard-to-learn sample. If the difference value of an anomaly image is greater than the first anomaly image and less than or equal to the second anomaly threshold, the anomaly image is determined to be a drift sample. If the difference value of an anomaly image is greater than the second anomaly image, the anomaly image is determined to be a novel type sample.

[0074] In this way, by following the above steps, we can determine which category of anomaly the abnormal image belongs to, so as to guide users in subsequent annotation.

[0075] Furthermore, in some embodiments, when the image to be detected is detected as an abnormal image, the method may further include:

[0076] The abnormal image is stored in the abnormal database;

[0077] When the number of abnormal images in the abnormal database reaches the update threshold, the target inference model is updated using the abnormal images in the abnormal database.

[0078] It should be noted that using an anomaly database to store abnormal images and updating the target inference model only when the number of abnormal images reaches a certain level is a common update strategy. Specifically, the target inference model can also be updated in real time.

[0079] Furthermore, in some embodiments, when the number of abnormal images in the abnormal database reaches a preset update threshold, the method further includes:

[0080] Based on the data annotation of the abnormal image, an annotated abnormal image is generated;

[0081] It should be noted that, to facilitate users' quick annotation of anomalous images, some preset options are provided for users to choose from. For difficult-to-learn samples and drifting samples, the user needs to be asked whether the prediction result is incorrect; for new types of samples, the user needs to be asked what the true category of the anomalous image is, and then the anomalous image is annotated.

[0082] It should also be noted that since hard-to-learn samples, drift samples, and novel samples are automatically classified based on an anomaly threshold, classification errors may occur. Therefore, the anomaly threshold can be designed to be adaptive. In other words, after the user labels the anomaly image with its true label, the anomaly threshold will also be adaptively adjusted based on the true label of the anomaly image.

[0083] Accordingly, in some embodiments, updating the original model features of the target inference model according to different proportions based on the anomaly category of the abnormal image includes:

[0084] Statistical analysis was performed on abnormal head images belonging to new types of samples, abnormal images belonging to difficult-to-learn samples, and abnormal images belonging to drift samples, and the data offset value was determined based on the statistical results.

[0085] If the data offset value is greater than the offset threshold, the target inference model is updated using the labeled anomaly image based on the first type of update strategy; or...

[0086] If the data offset value is less than or equal to the offset threshold, the target inference model is updated using the labeled anomaly image based on the second type of update strategy.

[0087] Wherein, the proportion of the original model features of the target inference model updated in the first type of update strategy is less than the proportion of the original model features of the target prediction model updated in the second type of update strategy.

[0088] It should be noted that if the data offset value is large, it means that the deviation between the image to be detected in production practice and the original training sample is greater, and the importance of the original model features is smaller. In this case, fewer original model features are retained during the update process. Conversely, if the data offset value is small, it means that the deviation between the image to be detected in production practice and the original training sample is smaller, and the importance of the original model features is greater. In this case, more original model features are retained during the update process.

[0089] For example, the first type of update strategy can be a few-sample update strategy, specifically a greedy clustering update strategy. That is, the target prediction model is trained on the neural network model using the original training samples. The target prediction model largely depends on the original training samples selected during the initial model training, and it can only be applied to images similar to the original training samples. However, in real-world production environments, the images to be detected are constantly changing, and it cannot be guaranteed that the original training samples can cover all types of images. Therefore, the target prediction model needs to be continuously updated to adapt to these changes.

[0090] In one specific embodiment, the first type of update strategy includes at least a greedy clustering update strategy, and the target inference model includes at least a memory module;

[0091] The step of updating the target inference model using the labeled anomaly image based on the first type of update strategy includes:

[0092] The original model features to be updated in the target inference model are stored in the memory module;

[0093] Based on the greedy clustering update strategy, the memory module is updated using the labeled abnormal images in the abnormal database.

[0094] It should be noted that the memory module can be the object updated by the dynamic update module. The memory module stores the dynamic parameters that need to be updated.

[0095] In another embodiment of this application, see Figure 2Based on a greedy clustering update strategy, updating the memory module using the labeled abnormal images from the abnormal database may include:

[0096] S201: Determine the incremental feature set based on the labeled abnormal images in the abnormal database;

[0097] It should be noted that the incremental feature set can be the incremental features of the abnormal image compared with the initial training image, or the incremental feature set can be the incremental feature set of the abnormal image labeled in the abnormal database compared with the initial training image.

[0098] S202: Obtain the current feature set of the memory module;

[0099] S203: Perform an intersection operation on the incremental feature set and the current feature set to obtain the target intersection;

[0100] S204: Cluster the target intersection to obtain a new feature set, and use the new feature set to update the parameters of the memory module.

[0101] It should be noted that during the parameter update process, only a portion of the knowledge feature vectors in the memory module (i.e., the new feature set) are updated. For example, taking the greedy clustering method as an example, the intersection M of all vectors of the incremental feature set of the current memory module and the new sample is first obtained, and then M is clustered. The number of clusters is the size of the original memory module. After the clustering is completed, the kernels of each category are used as the new memory module.

[0102] In this way, when an image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image. When the updated parameters are small, a greedy clustering update strategy can be used to update the model and obtain a new target inference model.

[0103] S103: If the image to be detected is a normal image, output the prediction result.

[0104] It should be noted that if the image to be detected is a normal image, the existing inference model can obtain an accurate prediction result, and then the prediction result can be output.

[0105] This application provides an inference method that inputs an image to be detected into a target inference model to obtain a prediction result. If the image to be detected is an anomalous image, the original model features of the target inference model are updated according to different proportions based on the anomalous category of the anomalous image. If the image to be detected is a normal image, the prediction result is output. This not only reduces computational resources and computational complexity but also improves the inference capability of the target inference model while maintaining its accuracy.

[0106] In another embodiment of this application, see [link to application]. Figure 3 It shows a schematic diagram of the working process of an inference device provided in an embodiment of this application, such as... Figure 3 As shown, the inference device may include a model inference module 301, an anomaly detection module 302, a user interaction module 303, and a dynamic update module 304.

[0107] Based on this inference device, this application embodiment uses an interactive online model update method, the specific working process of which is as follows:

[0108] 1) The inference device is connected to sensors in business scenarios (such as production lines) and can perform conventional model inference through the model inference module 301.

[0109] 2) During each inference process (or after inference), the anomaly detection module 302 will judge the input sample (i.e. the image to be detected). If it is judged as an anomaly sample (the image to be detected that has a large difference in distribution from the original training sample and has a low inference confidence), the sample and the inference result will be saved to the anomaly database in the inference device.

[0110] 3) When the number of abnormal images accumulates to a certain level, notify business users through a graphical interface or email, and guide them to annotate the data through the user interaction module 303 or provide feedback.

[0111] 4) When the user completes the interaction, the dynamic update module 304 updates the model using the currently labeled image based on the few-sample incremental metric learning algorithm.

[0112] It should be noted that the user interaction here mainly refers to data annotation, that is: annotating the abnormal images accumulated in step 3). After the annotation is completed, the model does not change. At this time, the user can choose to update the model and then proceed to step 4.

[0113] Regarding step 4), it's important to note that this primarily involves updating the dynamic parameters of the model (i.e., the memory module). However, directly using gradient descent for fine-tuning can easily lead to overfitting or catastrophic forgetting, meaning the target inference model's performance degrades on older tasks. To address this issue, the inference model must be based on a "Encoder + Decoder + Memory" structure (Memory-guided Encoder-Decoder, Mem Enc-Dec), where:

[0114] 1) The input to the Encoder is an image, and the output is the depth features of that image;

[0115] 2) The input to the Decoder is a measure of image features and knowledge features, and the output is the prediction result (such as a bounding box or mask);

[0116] 3) Memory is a collection of knowledge features.

[0117] When using the dynamic update module 304 for small sample incremental updates, the Encoder and Decoder remain unchanged, and only some knowledge feature vectors in the Memory are updated. The update strategy adopts a greedy clustering method, that is, first obtain the intersection M of all vectors of the feature set of the current Memory and the new sample, and then cluster M. The number of clusters is the size of the original Memory. After the clustering is completed, the kernel of each category is used as the new Memory.

[0118] In this embodiment, the target inference model is updated primarily through interactive updates and greedy clustering update strategies.

[0119] 1. Interactive Update: Using an edge computing box (i.e., an anomaly detection module 302) with a certain computing power, the device can detect abnormal samples and perform data annotation or feedback through interaction with the user.

[0120] 2. Greedy Clustering Update Strategy: Based on labeled data or given feedback information, the device dynamically updates the model through the dynamic update module 304. It constructs an incremental learning technique for small samples based on the greedy clustering method, thereby continuously improving the inference effect while avoiding catastrophic failures.

[0121] This application provides a reasoning method. Based on the above embodiments, the specific implementation of the aforementioned embodiments is described in detail. According to the technical solution of the aforementioned embodiments, in the actual application of the edge reasoning model, when there are certain differences between the input real data samples and the training set, the original feature vector of the target reasoning model is updated according to different proportions based on different anomaly types. In this way, the updated target reasoning model's accuracy significantly increases on new data, and its accuracy does not significantly decrease on the original data. More importantly, the model's parameters (or knowledge base size) do not increase indefinitely, and the required computing resources do not increase with each update. The reasoning ability of the updated model is significantly improved.

[0122] In another embodiment of this application, see [link to application]. Figure 4 This illustrates a schematic diagram of the composition structure of an inference device provided in an embodiment of this application. Figure 4 As shown, the inference device may include:

[0123] The inference unit 401 is configured to input the image to be detected into the target inference model to obtain the prediction result;

[0124] The update unit 402 is configured to update the original model features of the target inference model according to different proportions based on the abnormality category of the abnormal image when the image to be detected is an abnormal image.

[0125] The output unit 403 is configured to output the prediction result when the image to be detected is a normal image.

[0126] In some embodiments, the update unit 402 is specifically configured to calculate the difference value between the image to be detected and the initial training image through the anomaly detection module; and if the difference value is greater than the anomaly threshold, determine that the detection result indicates that the image to be detected is the anomaly image and determine the anomaly category of the anomaly image; and if the difference value is less than or equal to the anomaly threshold, determine that the detection result indicates that the image to be detected is the normal image.

[0127] In some embodiments, the update unit 402 is specifically configured to perform a measurement calculation on the abnormal image to determine the difference value of the abnormal image; and to determine the abnormal category of the abnormal image based on the difference value of the abnormal image and the difference threshold; wherein the abnormal category includes at least one of the following: new type sample, hard-to-learn sample, or drift sample.

[0128] In some embodiments, the update unit 402 is further configured to store the abnormal image in an abnormal database; and, if the number of abnormal images in the abnormal database reaches an update threshold, update the target inference model using the abnormal images in the abnormal database.

[0129] In some embodiments, the updating unit 402 is further configured to generate an annotated anomaly image based on data annotation of the anomaly image; and to perform statistics on anomaly images belonging to new type samples, anomaly images belonging to hard-to-learn samples, and anomaly images belonging to drift samples, and determine data offset values ​​based on the statistical results; and, if the data offset value is greater than an offset threshold, to update the target inference model using the annotated anomaly image based on a first type of update strategy; or, if the data offset value is less than or equal to an offset threshold, to update the target inference model using the annotated anomaly image based on a second type of update strategy; wherein the proportion of the original model features of the target inference model updated in the first type of update strategy is less than the proportion of the original model features of the target prediction model updated in the second type of update strategy.

[0130] In some embodiments, the first type of update strategy includes at least a greedy clustering update strategy, and the target inference model includes at least a memory module; the update unit 402 is specifically configured to store the original model features to be updated in the target inference model into the memory module; and based on the greedy clustering update strategy, update the memory module using the labeled abnormal images in the abnormal database.

[0131] In some embodiments, the update unit 402 is specifically configured to: determine an incremental feature set based on the labeled abnormal images in the abnormal database; obtain the current feature set of the memory module; perform an intersection operation on the incremental feature set and the current feature set to obtain a target intersection; perform clustering processing on the target intersection to obtain a new feature set, and use the new feature set to update the parameters of the memory module.

[0132] Understandably, in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular component. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0133] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] Therefore, this embodiment provides a computer storage medium storing a computer program that, when executed by at least one processor, implements the steps of the method described in any of the foregoing embodiments.

[0135] Based on the composition of the aforementioned inference device 40 and the computer storage medium, see [link to relevant documentation]. Figure 5 This illustrates a schematic diagram of the specific hardware structure of an electronic device provided in an embodiment of this application. For example... Figure 5 As shown, electronic device 50 may include: a communication interface 501, a memory 502, and a processor 503; the various components are coupled together via a bus system 504. It is understood that the bus system 504 is used to implement communication between these components. In addition to a data bus, the bus system 504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 All buses are labeled as bus system 504. Among them, communication interface 501 is used for receiving and sending signals during information exchange with other external network elements;

[0136] Memory 502 is used to store computer programs that can run on processor 503;

[0137] Processor 503, when running the computer program, performs the following:

[0138] The image to be detected is input into the target inference model to obtain the prediction result;

[0139] If the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image.

[0140] If the image to be detected is a normal image, the prediction result is output.

[0141] It is understood that the memory 502 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0142] The processor 503 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 503 or by instructions in software form. The processor 503 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 502, and the processor 503 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0143] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0144] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0145] Alternatively, as another embodiment, the processor 503 is further configured to perform the steps of the method described in any of the foregoing embodiments when running the computer program.

[0146] In some embodiments, see Figure 6 This illustrates a schematic diagram of the structural composition of an electronic device 50 provided in an embodiment of this application. For example... Figure 6 As shown, the electronic device 50 includes at least the inference device 40 described in any of the foregoing embodiments.

[0147] In this embodiment, for the electronic device 50, the image to be detected is input into the target inference model to obtain a prediction result. If the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image. If the image to be detected is a normal image, the prediction result is output. In this way, by using abnormal images to partially update the original features of the target inference model, not only are computational resources reduced and computational complexity lowered, but the inference ability of the target inference model can also be improved while ensuring its accuracy.

[0148] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0149] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0150] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0151] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0152] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A reasoning method, the method comprising: The image to be detected is input into the target inference model to obtain the prediction result; If the image to be detected is an abnormal image, the original model features of the target inference model are updated according to different proportions based on the abnormality category of the abnormal image. The abnormality category includes at least one of the following: new type of sample, hard-to-learn sample, or drift sample. The original model features include the model's parameters and / or feature vectors. If the image to be detected is a normal image, the prediction result is output; The process of updating the original model features of the target inference model at different ratios includes: statistically analyzing abnormal avatars belonging to new types of samples, abnormal images belonging to difficult-to-learn samples, and abnormal images belonging to drift samples, and determining data offset values ​​based on the statistical results. The data offset values ​​are inversely proportional to the ratio of the update of the original model features of the target inference model.

2. The method according to claim 1, further comprising: The difference between the image to be detected and the initial training image is calculated using the anomaly detection module; If the difference value is greater than the abnormal threshold, then the detection result indicates that the image to be detected is an abnormal image, and the abnormality category of the abnormal image is determined. If the difference value is less than or equal to the abnormal threshold, then the detection result indicates that the image to be detected is the normal image.

3. The method according to claim 2, wherein determining the anomaly category of the abnormal image includes: The abnormal image is measured and calculated to determine the difference value of the abnormal image; The anomaly category of the abnormal image is determined based on the difference between the difference value of the abnormal image and the anomaly threshold.

4. The method according to claim 3, wherein when the image to be detected is an abnormal image, the method further comprises: The abnormal image is stored in the abnormal database; When the number of abnormal images in the abnormal database reaches the update threshold, the target inference model is updated using the abnormal images in the abnormal database.

5. The method according to any one of claims 1-4, wherein if the number of abnormal images in the abnormal database reaches a preset update threshold, the method further comprises: Based on the data annotation of the abnormal image, an annotated abnormal image is generated; Accordingly, updating the original model features of the target inference model according to different proportions based on the anomaly category of the abnormal image includes: If the data offset value is greater than the offset threshold, the target inference model is updated using the labeled anomaly image based on the first type of update strategy; or... If the data offset value is less than or equal to the offset threshold, the target inference model is updated using the labeled anomaly image based on the second type of update strategy. Wherein, the proportion of the original model features of the target inference model updated in the first type of update strategy is less than the proportion of the original model features of the target prediction model updated in the second type of update strategy.

6. The method according to claim 5, wherein the first type of update strategy includes at least a greedy clustering update strategy, and the target inference model includes at least a memory module; The step of updating the target inference model using the labeled anomaly image based on the first type of update strategy includes: The original model features to be updated in the target inference model are stored in the memory module; Based on the greedy clustering update strategy, the memory module is updated using the labeled abnormal images in the abnormal database.

7. The method according to claim 6, wherein updating the memory module using the labeled abnormal images in the abnormal database based on the greedy clustering update strategy comprises: Based on the labeled abnormal images in the abnormal database, an incremental feature set is determined; Obtain the current feature set of the memory module; Perform an intersection operation on the incremental feature set and the current feature set to obtain the target intersection; Clustering is performed on the target intersection to obtain a new feature set, and the memory module is updated with the parameters of the new feature set.

8. A reasoning device, comprising: The inference unit is configured to input the image to be detected into the target inference model to obtain the prediction result; The update unit is configured to, when the image to be detected is an abnormal image, update the original model features of the target inference model according to different proportions based on the abnormality category of the abnormal image, wherein the abnormality category includes at least one of the following: new type sample, hard-to-learn sample, or drift sample, and the original model features include model parameters and / or feature vectors; updating the original model features of the target inference model according to different proportions includes: statistically analyzing abnormal avatars belonging to new type samples, abnormal images belonging to hard-to-learn samples, and abnormal images belonging to drift samples, and determining a data offset value based on the statistical results, wherein the data offset value is inversely proportional to the proportion of updating the original model features of the target inference model; The output unit is configured to output the prediction result when the image to be detected is a normal image.

9. An electronic device, comprising: Memory is used to store computer programs that can run on a processor; A processor, configured to perform the method as described in any one of claims 1 to 7 when running the computer program.

10. A computer storage medium storing a computer program that, when executed by at least one processor, implements the method as claimed in any one of claims 1 to 7.

Images