Visual inspection system, method and device, edge end, server and medium

Abstract

The embodiment of the invention provides a visual detection system, method and device, an edge end, a server and a medium, and relates to the technical field of visual detection, the visual detection system comprises the server and at least one edge end, the server is deployed with a basic model, the basic model has visual detection capability, each edge end is deployed with a visual processing model, and the visual processing model has visual detection capability. Each visual processing model is obtained by carrying out model parameter migration on the basic model by the server, and the edge end can be used for collecting a to-be-detected image of the target object, inputting the to-be-detected image into the self-deployed visual processing model and outputting a visual detection result. The basic model has the visual detection capability, so that the visual processing model obtained by carrying out model parameter migration on the basic model also has the visual detection capability. Therefore, the visual processing model does not need to be trained through an edge end with limited computing power, the dependence on equipment hardware is reduced, and the training efficiency of the visual processing model can be improved.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a vision inspection system, method, apparatus, edge device, server, and medium. Background Technology

[0002] Industrial vision inspection is an automated inspection technology centered on machine vision. Edge processing uses its own deployed vision processing model for visual inspection, commonly used in scenarios such as defective product detection and workpiece counting. Currently, related technologies require the edge processing unit to train the vision processing model on a sample set to obtain a trained model.

[0003] Due to limitations in computing power at the edge, the above methods suffer from low training efficiency for visual processing models. For example, training a visual processing model at the edge can take several days, impacting operational efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a visual inspection system, method, apparatus, edge computing device, server, and medium to improve the training efficiency of visual processing models. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a visual inspection system, the system comprising a server and at least one edge device. The server deploys a base model, the base model having visual inspection capabilities. Each edge device deploys a visual processing model, each visual processing model being obtained by the server through model parameter migration of the base model, wherein:

[0006] The edge end is used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result.

[0007] Optionally, the base model is trained based on sample images collected from all visual detection scenarios, and the visual processing model is trained based on sample images collected from fixed detection scenarios.

[0008] Optionally, the base model and the visual processing model have the same model structure, and the server is specifically used to: transfer the model parameters of the trained base model to the visual processing model according to the correspondence between the model structure of the base model and the model structure of the visual processing model, so as to obtain the visual processing model.

[0009] Optionally, the visual processing model is obtained by model distillation of the base model.

[0010] Optionally, the server is further configured to: acquire difficult samples collected by the edge during the visual detection process; and incrementally train the base model using the difficult samples and the corresponding result labels to obtain the incrementally trained base model.

[0011] Optionally, the server is further configured to: for the visual processing model, obtain the difficult samples collected during the visual detection process at the edge of the visual processing model; and perform personalized training on the basic model using the difficult samples corresponding to the visual processing model and the result labels corresponding to the difficult samples to obtain the basic model corresponding to the visual processing model.

[0012] Optionally, the edge end is further used to: determine difficult samples from the image to be detected acquired by itself; and / or to obtain difficult samples determined by the user from the image to be detected acquired by the edge end.

[0013] Optionally, the server is further configured to: when the number of difficult samples reaches a preset number, and / or, at preset intervals, incrementally train and / or personalized train the base model using the difficult samples and the result labels corresponding to the difficult samples.

[0014] Optionally, the edge end is further configured to: identify difficult samples from the images to be detected acquired by itself; and update the model parameters of the visual processing model deployed by itself online using the difficult samples and the result labels corresponding to the difficult samples, so as to obtain the online updated visual processing model.

[0015] Optionally, the edge endpoint is specifically used to update the model parameters of its deployed visual processing model online using any of the following methods:

[0016] From the unquantized model parameters included in the visual processing model, model parameters for visual detection of the target object are determined as model parameters to be adjusted; the difficult sample is input into the visual processing model so that the visual processing model performs visual detection on the difficult sample based on the current model parameters and outputs a prediction result; the model parameters to be adjusted are adjusted based on the difference between the prediction result and the result label corresponding to the difficult sample.

[0017] Extract the sample features of the difficult samples, and construct a registration sample by combining the sample features and the corresponding result labels; import the registration sample into the visual processing model so that the visual processing model can learn the correspondence between the sample features and the corresponding result labels through a meta-learning mechanism;

[0018] Extract the histogram features of the difficult samples, and import the histogram features and the corresponding result labels into the visual processing model so that the visual processing model can fuse the histogram features corresponding to the result labels with the deep learning features corresponding to the result labels that have been learned.

[0019] Optionally, the server is a local server deployed locally in the visual inspection scenario.

[0020] Secondly, embodiments of this application provide a visual inspection method applied to the edge of a visual inspection system. The visual inspection system further includes a server, on which a base model is deployed. The base model has visual inspection capabilities. The edge is equipped with a visual processing model, which is obtained by the server through model parameter migration of the base model. The method includes:

[0021] Acquire the image of the target object to be detected;

[0022] The image to be detected is input into the visual processing model deployed on its own, and the visual detection result is output.

[0023] Thirdly, embodiments of this application provide a visual inspection device applied at the edge of a visual inspection system. The visual inspection system further includes a server, on which a basic model is deployed. The basic model has visual inspection capabilities. A visual processing model is deployed at the edge, and the visual processing model is obtained by the server through model parameter migration of the basic model. The device includes:

[0024] The image acquisition module is used to acquire the image of the target object to be detected;

[0025] The result acquisition module is used to input the image to be detected into its own deployed visual processing model and output the visual detection result.

[0026] Fourthly, embodiments of this application provide an edge terminal, wherein a visual processing model is deployed on the edge terminal, a server is deployed on the visual detection system where the edge terminal is located, a basic model is deployed on the server, the basic model has visual detection capabilities, and the visual processing model is obtained by the server through model parameter migration of the basic model, wherein: the edge terminal is used to execute the steps of the method described in the second aspect above.

[0027] Fifthly, embodiments of this application provide a server, the server having a base model deployed thereon, the base model having visual inspection capabilities, and the visual inspection system on which the server is located further includes at least one edge terminal as described in the fourth aspect above.

[0028] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the second aspect above.

[0029] Beneficial effects of the embodiments in this application:

[0030] In the solution provided in this application embodiment, the visual inspection system may include a server and at least one edge terminal. The server deploys a base model with visual inspection capabilities. Each edge terminal deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge terminal can be used to acquire images of the target object to be inspected, input the images to be inspected into its own deployed visual processing model, and output the visual inspection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge terminal has weaker performance, so the base model cannot be deployed on the edge terminal; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual inspection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual inspection capabilities to obtain each visual processing model. Since the base model itself has visual inspection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual inspection capabilities. In this way, it is not necessary to train the visual processing model through the edge terminal with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0031] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0033] Figure 1 This is a schematic diagram of the structure of a visual inspection system provided in an embodiment of this application;

[0034] Figure 2 For based on Figure 1 The illustrated embodiment is a flowchart of incremental training of a base model.

[0035] Figure 3 For based on Figure 1 The illustrated embodiment is a flowchart of a personalized training process for a base model.

[0036] Figure 4 For based on Figure 1 The illustrated embodiment is a flowchart of an online update of a visual processing model.

[0037] Figure 5 For based on Figure 1 Another structural schematic diagram of the visual inspection system of the embodiment shown;

[0038] Figure 6 A flowchart illustrating a visual inspection method provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of a visual inspection device provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0041] Current technologies employ federated learning to update multiple vision processing models. This means that each edge device trains its vision processing model locally and then only uploads updated model parameters, achieving collaborative training while protecting data privacy. However, using federated learning in current technologies leads to the isolation of training data used by each vision processing model, with only model parameters being shared.

[0042] Unlike existing related technologies, this application has the following technical advantages:

[0043] First, the base model is trained using sample images collected from all visual detection scenarios, thereby maximizing the visual detection capability of the base model. These sample images can include difficult samples and / or normal samples; normal samples are the images to be detected that show normal visual detection results. Furthermore, the model parameters of the base model are transferred to the visual processing model, which further enhances the visual detection capability of the visual processing model.

[0044] In other words, the base model is first trained uniformly using all incremental data, and then each vision processing model is collaboratively optimized. The base model and the vision processing model share all the abnormal and difficult samples, which makes the base model more generalizable and robust in training. Through data consistency training, collaborative optimization between the base model and the vision processing model is achieved. The model structures of the base model and the vision processing model can be identical, which can improve the speed of model parameter transfer.

[0045] Secondly, the server in this application can be deployed either locally within the visual inspection scene or in the cloud. When the server is deployed locally, data privacy can be guaranteed through hardware encapsulation, as both the server and the edge device are localized. Furthermore, the edge device can perform visual inspection of target objects using its own deployed visual processing model, without needing to send the acquired images to the cloud server for visual inspection and then transmit the results back to the local visual inspection scene. This allows the entire computation to occur within the industrial site, eliminating cloud dependency, reducing communication latency, and improving the real-time performance of visual inspection.

[0046] Third, this application supports edge-based online learning with small sample sizes. This means the visual processing model can be fine-tuned using a small number of sample images, completing iterations within minutes or even less, without the need for conventional batch training that takes hours or even days. This allows for rapid fine-tuning and iteration. During production line changeovers, the visual processing model can be fine-tuned using a small number of sample images collected on the new production line, thus quickly adapting to the changeover. Furthermore, since the number of sample images used for fine-tuning is typically small, the required computing power is also lower, reducing reliance on hardware devices and enabling the process to be completed on embedded terminals such as edge devices with limited computing power.

[0047] Fourth, this application can dynamically collect difficult samples and / or normal samples during the visual inspection process, thereby continuously updating the sample library and realizing continuous optimization of the basic model and visual processing model.

[0048] To improve the training efficiency of visual processing models, embodiments of this application provide a visual inspection system, method, apparatus, edge device, server, computer-readable storage medium, and computer program product. The visual inspection system provided in this application embodiment is described below.

[0049] like Figure 1As shown, a visual inspection system includes a server 101 and at least one edge device 102. The server 101 is deployed with a base model that has visual inspection capabilities. Each edge device 102 is deployed with a visual processing model, and each visual processing model is obtained by the server 101 through model parameter migration of the base model.

[0050] The edge terminal 102 is used to acquire the image to be detected of the target object, input the image to be detected into its own deployed visual processing model, and output the visual detection result.

[0051] As can be seen, in this embodiment, the visual detection system may include a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge device can be used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the base model cannot be deployed on the edge device; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual detection capabilities to obtain each visual processing model. Since the base model itself has visual detection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual detection capabilities. In this way, it is not necessary to train the visual processing model through the edge device with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0052] Visual inspection scenarios refer to environments where visual inspection is required, such as industrial production lines, goods sorting areas, and warehouses. First, let's introduce the hardware architecture of a visual inspection scenario:

[0053] In a visual inspection scenario, at least one edge device 102 can be deployed. Edge device 102 can include one or more edge devices, such as processing devices, smart cameras, etc. If edge device 102 includes multiple edge devices, it can consist of a processing device and a smart camera. The processing device can be connected to the smart camera to process images captured by the smart camera, run various visual processing models, etc. Specifically, it can be a PC (Personal Computer), processor, acquisition card, etc., without specific limitations. Server 101 can be a physical computing device physically deployed locally in the visual inspection scenario, such as an industrial control computer, rack-mounted server, edge computing server, etc. Server 101 can also be deployed in the cloud; both are reasonable options.

[0054] Server 101 can be deployed with a basic model that has visual detection capabilities. This basic model, also known as a large model, is designed for general scenarios and typically has a complex structure, consumes a large amount of memory, requires a long training time, and has strong generalization ability. In one implementation, the basic model can be pre-trained on general datasets such as COCO (Common Objects in Context) or industry-specific datasets, and pre-installed on the server at the time of shipment.

[0055] This application does not limit the type of the base model. For example, the base model can be a model with visual detection capabilities, such as YOLO (YouOnly Look Once), DINO (DETR with Improved DeNoising Anchor Boxes), or AnomalyGPT (Anomaly Detection Generative Pre-trained Transformer). This is all reasonable.

[0056] The aforementioned edge device 102 can be a hardware device integrating image acquisition, processing, and AI (Artificial Intelligence) algorithms, capable of performing visual tasks such as object detection, defective product detection, and workpiece counting locally in real time. The edge device 102 can be deployed with a visual processing model. A visual processing model, also known as a lightweight model or small model, typically has a simplified structure, low memory footprint, and short training time. In the case where the edge device 102 is a smart camera, the aforementioned visual processing model can be a camera model.

[0057] Since server 101's performance is sufficient to support the operation of the basic model, it can be deployed on server 101. Compared to server 101, edge device 102 has weaker performance, therefore the basic model cannot be deployed on edge device 102; instead, a visual processing model is deployed there. To ensure that the visual processing model also possesses strong visual detection capabilities, during the deployment phase of the visual processing model, server 101 can perform model parameter migration on the basic model to obtain at least one visual processing model.

[0058] After the server 101, the basic model, the edge device 102, and the vision processing model are deployed, visual inspection can be performed through the edge device 102. Specifically, the edge device 102 can acquire the image of the target object to be inspected and input the image into its deployed vision processing model, which can then output the visual inspection result corresponding to the image. The target object can be any object that needs to be visually inspected, such as a workpiece, a material bin, etc.

[0059] The above process will be illustrated with examples of specific visual inspection types: For instance, when it is necessary to count workpieces through the edge, the edge can take a picture of the workpiece to be inspected and input the picture into its own deployed visual processing model, which will then output the number of workpieces.

[0060] For example, when it is necessary to inspect workpieces for defects through the edge, the edge can take an image of the workpiece to be inspected and input the image into its own deployed vision processing model. The vision processing model will then output whether the workpiece is defective and, if the workpiece is defective, output the location of the defect.

[0061] Generally, server 101 does not perform visual detection using the base model. Instead, during visual detection at the edge 102, it updates and optimizes the base model to improve the overall capability of the visual detection system. The visual processing model is also updated using the model parameters of the base model to ensure sufficient training. This process will be described in detail in subsequent embodiments and will not be elaborated here. Of course, in some cases, server 101 may also perform visual detection using the base model.

[0062] As can be seen, in this embodiment, since the server's performance can support the operation of the basic model, the basic model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the basic model cannot be deployed on the edge device; instead, a visual processing model is deployed. To ensure that the visual processing model also has strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter migration on the basic model with visual detection capabilities to obtain the visual processing model, without needing to train the visual processing model through the edge device with limited computing power. This reduces dependence on device hardware and improves the training efficiency of the visual processing model. During the visual detection process, the edge device can perform visual detection on the target object using its own deployed visual processing model. Since the visual detection process is completely independent of the cloud, cloud communication latency during visual detection can be eliminated, improving the real-time performance of visual detection. Through the closed collaborative architecture of the server and the edge device, both model capacity and computing resources can be balanced.

[0063] Furthermore, with servers and edge devices deployed entirely locally, they can form a closed system, and the data generated during the visual inspection process never leaves the industrial site. Therefore, data privacy can be guaranteed and data security requirements can be met through hardware enclosure.

[0064] As one implementation of this application, the model structure of the basic model and the visual processing model can be the same. In this case, the server can specifically be used for:

[0065] Based on the correspondence between the model structure of the base model and the model structure of the visual processing model, the model parameters of the trained base model are transferred to the visual processing model to obtain the visual processing model.

[0066] Because the base model and the vision processing model have the same model structure, the server can accurately determine the correspondence between the neurons, dimensions, and functional modules in each network layer. Thus, the server can transfer the model parameters of the trained base model (used for vision detection) to the corresponding network layers of the vision processing model, based on the correspondence between their model structures, to obtain the vision processing model.

[0067] In one implementation, the visual processing model can be obtained by model distillation of the base model. In this case, the server can perform model distillation and quantization on the trained base model, and then send the model parameters of the trained base model to the visual processing model according to the correspondence between the model structures of the base model and the visual processing model.

[0068] Model distillation is a technique for transferring knowledge from a complex model to a simplified model, achieving teacher-student model compression by softening the output probability distribution. In this embodiment, the complex model is the base model, and the simplified model is the visual processing model. Quantization is a technique for converting neural network parameters from high precision to low precision, such as converting model parameters from 32-bit floating-point to 8-bit integers, thereby reducing computational resource consumption and improving inference speed.

[0069] After obtaining the visual processing model, the edge processing unit can also adaptively prune the model to generate a deployable lightweight model. Specifically, the edge processing unit can adaptively prune the visual processing model through at least one of the following two implementation methods:

[0070] In the first implementation, the edge device can perform channel pruning on the visual processing model according to its own hardware specifications, so that the complexity of the pruned visual processing model matches the hardware capacity of the edge device. Specifically, the hardware specifications can be the computing power of an NPU (Neural Processing Unit).

[0071] In the second implementation, the edge can determine the non-sensitive layers of the updated visual processing model by performing quantized perceptual training on the updated model, and then quantize the model parameters of the non-sensitive layers. For example, the precision of the model parameters of the non-sensitive layers can be adjusted to 8 bits (binary digit), while the precision of the model parameters of the sensitive layers can be retained at 16 bits, achieving mixed precision.

[0072] As can be seen, in this embodiment, the server can transfer the model parameters of the trained base model to the visual processing model according to the correspondence between the model structure of the base model and the model structure of the visual processing model, thus obtaining the visual processing model. Since the base model and the visual processing model have the same model structure, that is, they have a consistent training framework, the server can quickly and accurately determine the correspondence between the neuron connections, dimensional configurations, and functional modules of the two models in each network layer, thereby improving the transfer speed of model parameters.

[0073] As one implementation method of this application, such as Figure 2 As shown, the server described above can also be used to execute steps S201 and S202:

[0074] S201, Obtain the difficult samples collected during the visual inspection process at the edge end.

[0075] During visual inspection, the edge device can acquire images to be inspected and identify images whose visual inspection results meet the abnormal conditions from its own acquired images as hard samples. The server can then obtain these hard samples.

[0076] Difficult samples can be images to be detected whose visual detection results meet abnormal conditions. Abnormal conditions can be that the visual detection results are inconsistent with the actual results, or that the confidence level of the visual detection results is less than a first threshold. The first threshold can be set according to actual needs, such as 0.6, 0.5, 0.4, etc., and is not specifically limited here.

[0077] S202, using the difficult samples and the corresponding result labels, the basic model is incrementally trained to obtain the incrementally trained basic model.

[0078] Since edge detection performs poorly on difficult samples, to improve the overall capability of the visual detection system, the server can incrementally train the base model using difficult samples and their corresponding result labels to obtain an incrementally trained base model. The result labels can be assigned by the user based on the actual results of the difficult samples. The difficult samples used for incremental training of the base model can be: at least one edge-based image that meets the anomaly conditions and is to be detected.

[0079] In one implementation, incremental training of the base model can be performed by inputting a difficult sample into the base model, allowing the base model to determine the prediction result corresponding to the difficult sample based on its current model parameters. The server can then adjust the model parameters of the base model through backpropagation based on the difference between the prediction result and the label corresponding to the difficult sample, until the base model meets the convergence condition, resulting in an incrementally trained base model. Alternatively, the server can also perform incremental training on the base model using both difficult and normal samples. Unless otherwise specified, subsequent embodiments involving model training can utilize the methods described herein.

[0080] Next, the server can transfer the model parameters of the incrementally trained base model to the vision processing model to improve the vision processing model's subsequent vision detection capabilities.

[0081] Since the actual visual detection is performed by the visual processing model, not the base model, even if incremental training takes a long time, it will not affect the normal operation of visual detection. In other words, the evolution cycle of the base model is not under pressure from on-site operation. In view of this, in one implementation, the difficult samples used for incremental training of the base model can be the entire incremental data. The entire incremental data is all the images to be detected collected from all edges from the last incremental training to the current time.

[0082] As can be seen, in this embodiment of the application, during the visual detection process, the server can acquire difficult samples collected by the edge devices during the visual detection process. These difficult samples are images to be detected whose corresponding visual detection results satisfy abnormal conditions. Using the difficult samples and their corresponding result labels, the base model is incrementally trained to obtain the incrementally trained base model. Since the difficult samples used for incremental training of the base model can be collected by multiple edge devices, and the types of visual detection performed by different edge devices may be different, and the detected target objects may also be different, sharing data from the difficult samples collected by multiple edge devices during the incremental training of the base model can make the base model more generalizable.

[0083] As one implementation method of this application, such as Figure 3 As shown, the server described above can also be used to execute steps S301 and S302:

[0084] S301, For the visual processing model, obtain the difficult samples collected during the visual detection process at the edge of the visual processing model.

[0085] In order to personalize the visual processing model, the server can obtain the difficult samples collected during the visual detection process for one or more visual processing models at the edge of that visual processing model.

[0086] S302, using the difficult samples corresponding to the visual processing model and the result labels corresponding to the difficult samples, the basic model is trained in a personalized manner to obtain the basic model corresponding to the visual processing model.

[0087] After obtaining the difficult samples corresponding to the visual processing model, the server can perform personalized training on the base model for one or more visual processing models, using the difficult samples and their corresponding result labels. This results in a base model for that visual processing model. Next, the server can transfer the model parameters of the personalized trained base model to the visual processing model, thus achieving personalized fine-tuning of the visual processing model. Of course, personalized training can also use normal samples with normal detection results on top of the difficult samples, which is also reasonable.

[0088] For example, suppose visual processing models 1 to 3 belong to edge 1 to edge 3 respectively. Edge 1 collects difficult samples 1 to 5, edge 2 collects difficult samples 6 to 10, and edge 3 collects difficult samples 11 to 15. Then, the server can use difficult samples 1 to 5 to perform personalized training on the base model, obtaining a base model 1 specifically for visual processing model 1. Similarly, the server can use difficult samples 6 to 10 to perform personalized training on the base model, obtaining a base model 2 specifically for visual processing model 2, and use difficult samples 11 to 15 to perform personalized training on the base model, obtaining a base model 3 specifically for visual processing model 3. Next, the server can transfer the model parameters of base model 1 to visual processing model 1, the model parameters of base model 2 to visual processing model 2, and the model parameters of base model 3 to visual processing model 3.

[0089] As can be seen, in this embodiment, the server can obtain difficult samples collected by the edge device to which the visual processing model belongs during the visual detection process for one or more visual processing models. Using the difficult samples corresponding to the visual processing model and the result labels corresponding to those difficult samples, the server can perform personalized training on the basic model to obtain the basic model corresponding to the visual processing model. In this way, by individually fine-tuning each visual processing model, a more fully trained visual processing model can be provided. Currently, in related technologies, traditional edge devices use a single embedded model, which suffers from problems such as limited model capacity leading to high false negative rates and inability to adapt to scene changes. For example, in precision industrial fields such as electronic product assembly and automotive parts manufacturing, due to the significant differences in the types of objects being detected, shooting angles, and lighting at different workstations, embedded models are limited by computational accuracy, resulting in insufficient generalization and recognition capabilities for various scenes, leading to a persistently high overall false negative rate. Unlike current related technologies, in this embodiment, one or more visual processing models can be continuously fine-tuned individually, improving the stability of the visual processing model at the edge device in its respective application scenario. This is particularly suitable for fixed-scene visual inspection systems such as industrial quality inspection that require long-term stable operation.

[0090] As one embodiment of this application, the aforementioned edge terminal can also be used to determine difficult samples through at least one of the following two methods:

[0091] From the image to be detected acquired by itself, identify difficult samples; and / or, acquire difficult samples identified by the user from the image to be detected acquired from the edge.

[0092] In the first implementation, the edge processing unit can identify difficult samples from the images it has acquired for detection. As a specific implementation, if the confidence level of the visual detection result is low, it indicates significant uncertainty in the result, meaning the visual processing model's detection capability is insufficient. Therefore, the edge processing unit can identify images from its acquired images where the confidence level of the visual detection result is less than a first threshold, designating these as difficult samples, and upload them to the server. The confidence level can be output by the visual processing model along with the visual detection result.

[0093] In the second implementation, the user can identify difficult samples, such as those that have been missed or falsely detected, from the images to be detected captured by the edge device. Then, the edge device can acquire the difficult samples identified by the user and upload them to the server.

[0094] After identifying difficult samples using the methods described above, the edge application can continuously build a difficult sample library based on these samples. In this case, the edge application can upload the difficult sample library to the server. Of course, the difficult sample library can also include normal samples, which is perfectly reasonable.

[0095] As can be seen, in this embodiment, the edge device can identify difficult samples from the image to be detected that it has acquired itself; and / or, acquire difficult samples identified by the user from the image to be detected acquired by the edge device. In this way, difficult samples can be acquired through both user manual annotation and automatic acquisition by the edge device, achieving dual-channel acquisition. The edge device can upload difficult samples to the server, which can then use these samples to uniformly optimize the base model. Through the evolutionary mechanism of difficult samples, the base model is continuously iterated, thereby continuously optimizing the performance of the entire visual detection system.

[0096] As one embodiment of this application, the server described above can also be used for:

[0097] When the number of difficult samples reaches a preset number, and / or, every preset time interval, incremental training and / or personalized training are performed on the base model using the difficult samples and the result labels corresponding to the difficult samples.

[0098] During the visual detection process, the edge continuously identifies difficult samples. When the number of difficult samples reaches a preset threshold, it indicates that a sufficient number of difficult samples have been accumulated. In this case, the server can train the base model. The preset threshold can be set according to actual needs, such as 50, 100, 150, etc., without specific limitations here.

[0099] In addition, the interval for training the base model can be preset, i.e., the preset duration. The preset duration can be set according to actual needs, such as 8 hours, 12 hours, 24 hours, etc., without specific limitations here. The server can also train the base model at preset intervals.

[0100] The server trains the base model, specifically through incremental training and / or personalized training. In other words, in the first case, the server can perform only incremental training on the base model; in the second case, the server can perform only personalized training on the base model; and in the third case, the server can perform incremental training on the base model first, followed by personalized training. Since incremental training can use all anomalous samples, and personalized training can use difficult samples collected from the edge of the target visual processing model, both the versatility and specificity of the visual processing model can be balanced through incremental and personalized training. The specific methods of incremental and personalized training have been described in the preceding embodiments and will not be repeated here.

[0101] As can be seen, in this embodiment, the server can also perform incremental training and / or personalized training on the basic model using the difficult samples and their corresponding result labels at preset intervals when the number of difficult samples reaches a preset number. This allows the performance of the visual inspection system to continuously evolve.

[0102] As one implementation method of this application, such as Figure 4 As shown, the aforementioned edge end can also be used to perform steps S401 and S402:

[0103] S401 identifies difficult samples from the images to be detected acquired by itself.

[0104] The edge processing unit can identify difficult samples from the images it collects for detection. In one implementation, a low confidence level in the visual detection result indicates significant uncertainty. To optimize the visual processing model, the edge processing unit can collect samples with poor detection performance during the visual detection process. Specifically, the edge processing unit can identify images from its collected images where the confidence level of the corresponding visual detection result is lower than a second threshold as difficult samples. The second threshold can be set according to actual needs, such as 0.6, 0.5, 0.4, etc., and is not specifically limited here.

[0105] S402, using the difficult samples and the corresponding result labels, the model parameters of the visual processing model deployed by itself are updated online to obtain the updated visual processing model.

[0106] After identifying the difficult samples, users can label them, thus determining the corresponding result labels. Next, the edge processing unit can use the difficult samples and their corresponding result labels to update the model parameters of its deployed visual processing model online, resulting in an updated visual processing model.

[0107] Since edge devices typically only perform visual detection on fixed detection scenarios after deployment, the capability requirements for the visual processing model can be significantly reduced. Therefore, in one implementation, the number of difficult samples used for online updates to the visual processing model can be less than a preset sample number threshold. This threshold can be smaller than the preset number in the aforementioned embodiments and can be set according to actual needs, such as 10, 6, 5, etc. In this way, the visual processing model can learn from a small number of difficult samples, adapting to subtle changes within its specific scenario, thereby rapidly improving online detection performance and achieving rapid response to changes in scenarios and requirements. The visual processing model can be trained based on sample images collected from a fixed detection scenario. The fixed detection scenario is the scenario that the edge device to which the visual processing model belongs targets during visual detection. The sample images used to train the visual processing model can include difficult samples and / or normal samples.

[0108] As can be seen, in this embodiment, since the number of difficult samples is usually small, the computational power requirement for online updates is relatively low. Online updates can be performed directly on the edge device (i.e., the edge end), enabling real-time edge learning. Current related technologies require model updates through conventional batch training, resulting in a large number of samples and long update cycles, typically exceeding 24 hours or even reaching several days, making it impossible to quickly respond to scene changes. For example, when a production line switches product models, the false alarm rate of the original detection model surges due to model differences. After the model change, manual intervention is required to label a large number of samples and retrain the model, causing production capacity losses. Unlike current related technologies, the solution provided in this embodiment, after deployment, typically only performs visual detection on fixed scenes. Therefore, the capability requirements for the visual processing model can be significantly reduced. Small samples (i.e., a small number of difficult samples, such as 5 to 10 difficult samples) are used to update the visual processing model online, enabling small-sample learning. The visual processing model can be rapidly fine-tuned and iterated within minutes, allowing it to adapt to production line changes and improving the visual detection performance of the edge end at the current workstation.

[0109] As one embodiment of this application, the aforementioned edge terminal is specifically used to update the model parameters of its deployed visual processing model online using any of the following three implementation methods:

[0110] In a first embodiment, model parameters for visual detection of the target object are determined from the unquantized model parameters included in the visual processing model, and are used as model parameters to be adjusted; the difficult sample is input into the visual processing model so that the visual processing model performs visual detection on the difficult sample based on the current model parameters and outputs a prediction result; the model parameters to be adjusted are adjusted based on the difference between the prediction result and the result label corresponding to the difficult sample.

[0111] Specifically, the edge processing unit can perform sensitivity analysis on the unquantized model parameters included in the visual processing model, such as calculating the gradient contribution value of each model parameter in the visual detection task. Furthermore, it can determine the model parameters used for visual detection of the target object as model parameters to be adjusted, such as the convolutional kernel weights responsible for contour extraction, the bias terms of the fully connected layers responsible for class determination, and so on.

[0112] Next, the edge processing unit can input difficult samples into the visual processing model. This model can then perform visual detection on the difficult samples based on the current model parameters and output prediction results. The edge processing unit can then adjust the model parameters in the visual processing model based on the difference between the prediction results and the label of the difficult samples, until the visual processing model meets the convergence condition, resulting in an online updated visual processing model.

[0113] In the second embodiment, sample features of the difficult samples are extracted, and the sample features and the corresponding result labels are used to construct a registration sample. The registration sample is then imported into the visual processing model so that the visual processing model can learn the correspondence between the sample features and the corresponding result labels through a meta-learning mechanism.

[0114] Specifically, the edge processing unit can extract features from difficult samples to obtain their sample features, and construct a registration sample by combining these features and their corresponding result labels. Next, the edge processing unit can import these registration samples into the visual processing model. The visual processing model can employ a meta-learning architecture, optimizing results through registration query samples. In other words, the visual processing model can learn the correspondence between sample features and their corresponding result labels through a meta-learning mechanism. Consequently, the edge processing unit can obtain the online updated visual processing model.

[0115] In the third implementation, the histogram features of the difficult samples are extracted, and the histogram features and the corresponding result labels are imported into the visual processing model so that the visual processing model can fuse the histogram features corresponding to the result labels with the deep learning features corresponding to the result labels that have been learned.

[0116] Specifically, the edge processing unit can extract histogram features from hard samples and import these histogram features and their corresponding labels into the visual processing model. This model can then fuse the histogram features corresponding to the label with the learned deep learning features already associated with that label. For example, it can concatenate the histogram features with the learned deep learning features. This allows the edge processing unit to obtain the online-updated visual processing model. The histogram features can also be other non-deep learning features; no specific limitations are imposed here.

[0117] As can be seen, in this embodiment, the edge device can update the model parameters of its deployed visual processing model online using any of the methods described above. This allows for rapid fine-tuning of the visual processing model to adapt to production line changes.

[0118] As one embodiment of this application, another schematic diagram of the visual inspection system can be shown as follows: Figure 5 As shown, this includes server 501 and N edge terminals ( Figure 5Only edge terminal 1 (502), edge terminal 2 (503), and edge terminal N (504) are shown. Edge terminal 1 (502) is deployed in scene 1, edge terminal 2 (503) is deployed in scene 2, and edge terminal N (504) is deployed in scene N. The placement angle of the target object is different in each scene. Server 501 has a built-in basic model, and the edge terminals have built-in visual processing models.

[0119] During the visual detection of target objects, edge computing can identify difficult samples from the images it collects and use these difficult samples to perform small-sample online learning on its deployed visual processing model. Specifically, this can be achieved as follows: Figure 5 The dashed box in the lower left corner shows that the edge-end can perform supplementary learning on its deployed visual processing model using the difficult sample 505, i.e., online learning with small samples. This allows for targeted improvement of the visual processing model's detection capability for the current visual detection scenario. Supplementary learning updates the parameters of the visual processing model. Subsequently, the edge-end can perform visual detection on the image to be detected 506 using the updated visual processing model, obtaining visual detection result 507. Visual detection result 507 is superior to the visual detection result corresponding to the difficult sample 505.

[0120] During the visual detection of target objects, the edge devices can also collect sample images and upload them to server 501 via a bidirectional communication protocol. Specifically, edge device 1 502 can upload sample image 1, edge device 2 503 can upload sample image 2, and edge device N 504 can upload sample image N. Server 501 can incrementally train the base model using sample image 1, sample image 2, ..., sample image N, thereby evolving the base model.

[0121] Next, server 501 can perform personalized training and adaptive distillation on the incrementally trained base model using sample images uploaded by one or more edge devices. The resulting base model parameters are then transferred to the visual processing model at that edge device, thus achieving collaborative optimization between server 501 and the edge device. Specifically, server 501 can dynamically allocate the complexity of the visual processing model based on the NPU computing power of the edge device, achieving resource adaptation.

[0122] Specifically, for edge 1 502, server 501 can perform personalized training and adaptive distillation on the incrementally trained base model using sample image 1, and transfer the model parameters of the obtained base model to visual processing model 1; for edge 2 503, server 501 can perform personalized training and adaptive distillation on the incrementally trained base model using sample image 2, and transfer the model parameters of the obtained base model to visual processing model 2; for edge N 504, server 501 can perform personalized training and adaptive distillation on the incrementally trained base model using sample image N, and transfer the model parameters of the obtained base model to visual processing model N.

[0123] In the technical solution of this application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0124] Corresponding to the visual inspection system described above, this application also provides a visual inspection method, which will be described below.

[0125] like Figure 6 As shown, a visual inspection method is applied to the edge of a visual inspection system. The visual inspection system also includes a server, on which a base model is deployed. The base model has visual inspection capabilities. The edge is equipped with a visual processing model, which is obtained by the server through model parameter migration of the base model. The method includes:

[0126] S601, Acquire the image of the target object to be detected;

[0127] S602, the image to be detected is input into the visual processing model deployed by itself, and the visual detection result is output.

[0128] As can be seen, in this embodiment, the visual detection system may include a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge device can be used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the base model cannot be deployed on the edge device; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual detection capabilities to obtain each visual processing model. Since the base model itself has visual detection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual detection capabilities. In this way, it is not necessary to train the visual processing model through the edge device with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0129] As one embodiment of this application, the above method may further include:

[0130] From the image to be detected acquired by itself, identify difficult samples; and / or, acquire difficult samples identified by the user from the image to be detected acquired from the edge.

[0131] The above implementation methods have been described in detail above, so they will not be repeated here.

[0132] As can be seen, in this embodiment, the edge device can identify difficult samples from the image to be detected that it has acquired itself; and / or, acquire difficult samples identified by the user from the image to be detected acquired by the edge device. In this way, difficult samples can be acquired through both user manual annotation and automatic acquisition by the edge device, achieving dual-channel acquisition. The edge device can upload difficult samples to the server, which can then use these samples to uniformly optimize the base model. Through the evolutionary mechanism of difficult samples, the base model is continuously iterated, thereby continuously optimizing the performance of the entire visual detection system.

[0133] As one embodiment of this application, the above method may further include:

[0134] From the images to be detected collected by itself, difficult samples are identified; using the difficult samples and the corresponding result labels, the model parameters of the visual processing model deployed by itself are updated online to obtain the updated visual processing model.

[0135] The above implementation methods have been described in detail above, so they will not be repeated here.

[0136] As can be seen, in this embodiment, since the number of difficult samples is usually small, the computational power requirement for online updates is relatively low. Online updates can be performed directly at the edge (edge ​​end), enabling real-time edge learning. Currently, related technologies require model updates through conventional batch training, resulting in a large number of samples and long update cycles, typically exceeding 24 hours or even several days, making it impossible to quickly respond to scene changes. For example, when a production line switches product models, the false alarm rate of the original detection model surges due to model differences. After the model change, manual intervention is required to label a large number of samples and retrain the model, causing production capacity losses. Unlike current related technologies, the solution provided in this embodiment typically only performs visual detection on fixed scenes after deployment at the edge end. Therefore, the capability requirements for the visual processing model can be significantly reduced. Small samples (i.e., a small number of difficult samples, such as 5 to 10 difficult samples) are used for online updates of the visual processing model, enabling small-sample learning. The visual processing model can be rapidly fine-tuned and iterated within minutes, allowing it to adapt to production line changes and improving the visual detection performance of the edge end at the current workstation.

[0137] As one implementation of this application, the step of updating the model parameters of the deployed visual processing model online using the difficult samples and the corresponding result labels of the difficult samples can include any of the following methods:

[0138] From the unquantized model parameters included in the visual processing model, model parameters for visual detection of the target object are determined as model parameters to be adjusted; the difficult sample is input into the visual processing model so that the visual processing model performs visual detection on the difficult sample based on the current model parameters and outputs a prediction result; the model parameters to be adjusted are adjusted based on the difference between the prediction result and the result label corresponding to the difficult sample.

[0139] Extract the sample features of the difficult samples, and construct a registration sample by combining the sample features and the corresponding result labels; import the registration sample into the visual processing model so that the visual processing model can learn the correspondence between the sample features and the corresponding result labels through a meta-learning mechanism;

[0140] Extract the histogram features of the difficult samples, and import the histogram features and the corresponding result labels into the visual processing model so that the visual processing model can fuse the histogram features corresponding to the result labels with the deep learning features corresponding to the result labels that have been learned.

[0141] The above implementation methods have been described in detail above, so they will not be repeated here.

[0142] As can be seen, in this embodiment, the edge device can update the model parameters of its deployed visual processing model online using any of the methods described above. This allows for rapid fine-tuning of the visual processing model to adapt to production line changes.

[0143] Corresponding to the above-mentioned visual inspection method, this application also provides a visual inspection device, which will be described below.

[0144] like Figure 7 As shown, a visual inspection device is applied at the edge of a visual inspection system. The visual inspection system also includes a server, on which a base model is deployed. The base model has visual inspection capabilities. The edge device deploys a visual processing model, which is obtained by the server through model parameter migration of the base model. The device includes:

[0145] Image acquisition module 701 is used to acquire the image of the target object to be detected;

[0146] The result acquisition module 702 is used to input the image to be detected into its own deployed visual processing model and output the visual detection result.

[0147] As can be seen, in this embodiment, the visual detection system may include a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge device can be used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the base model cannot be deployed on the edge device; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual detection capabilities to obtain each visual processing model. Since the base model itself has visual detection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual detection capabilities. In this way, it is not necessary to train the visual processing model through the edge device with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0148] As one embodiment of this application, the above-described apparatus may further include:

[0149] The first sample determination module is used to determine difficult samples from the image to be detected acquired by itself; and / or to acquire difficult samples acquired by the user from the edge end.

[0150] As one embodiment of this application, the above-described apparatus may further include:

[0151] The second sample determination module is used to determine difficult samples from the images to be detected that it has acquired;

[0152] The model update module is used to update the model parameters of its deployed visual processing model online using the difficult samples and the corresponding result labels, so as to obtain the updated visual processing model.

[0153] As one embodiment of this application, the above-mentioned model update module may include:

[0154] The first update submodule is used to determine model parameters for visual detection of the target object from the unquantized model parameters included in the visual processing model, as model parameters to be adjusted; input the difficult sample into the visual processing model so that the visual processing model performs visual detection on the difficult sample based on the current model parameters and outputs a prediction result; and adjust the model parameters to be adjusted based on the difference between the prediction result and the result label corresponding to the difficult sample.

[0155] The second update submodule is used to extract the sample features of the difficult samples, construct the sample features and the corresponding result labels of the sample features as registered samples, and import the registered samples into the visual processing model so that the visual processing model can learn the correspondence between the sample features and the corresponding result labels of the sample features through a meta-learning mechanism.

[0156] The third update submodule is used to extract the histogram features of the difficult samples and import the histogram features and the corresponding result labels into the visual processing model, so that the visual processing model can fuse the histogram features corresponding to the result labels with the deep learning features corresponding to the already learned result labels.

[0157] This application embodiment also provides an edge terminal, wherein a visual processing model is deployed on the edge terminal, and a server is deployed on the visual detection system where the edge terminal is located. The server deploys a basic model, which has visual detection capabilities. The visual processing model is obtained by the server through model parameter migration of the basic model. The edge terminal is used to execute the steps of the visual detection method described in any of the above embodiments.

[0158] As can be seen, in this embodiment, the visual detection system may include a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge device can be used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the base model cannot be deployed on the edge device; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual detection capabilities to obtain each visual processing model. Since the base model itself has visual detection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual detection capabilities. In this way, it is not necessary to train the visual processing model through the edge device with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0159] This application also provides a server, which is deployed with a base model that has visual inspection capabilities. The visual inspection system on which the server is located also includes at least one edge terminal as described in any of the above embodiments.

[0160] As can be seen, in this embodiment, the visual detection system may include a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter transfer of the base model. The edge device can be used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result. Since the server's performance can support the operation of the base model, the base model can be deployed on the server. Compared to the server, the edge device has weaker performance, so the base model cannot be deployed on the edge device; instead, a visual processing model is deployed. To enable the visual processing model to have strong visual detection capabilities, during the deployment phase of the visual processing model, the server can perform model parameter transfer on the base model with visual detection capabilities to obtain each visual processing model. Since the base model itself has visual detection capabilities, the visual processing model obtained by performing model parameter transfer on the base model also has visual detection capabilities. In this way, it is not necessary to train the visual processing model through the edge device with limited computing power, reducing the dependence on device hardware and improving the training efficiency of the visual processing model. During the visual inspection process, the edge device can perform visual inspection of the target object through its own deployed visual processing model.

[0161] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described visual detection methods.

[0162] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the visual detection methods described above.

[0163] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0164] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0165] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for devices, edge devices, servers, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0166] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A visual inspection system, characterized in that, The system includes a server and at least one edge device. The server deploys a base model with visual detection capabilities. Each edge device deploys a visual processing model, which is obtained by the server through model parameter migration of the base model. The edge end is used to acquire the image of the target object to be detected, input the image to be detected into its own deployed visual processing model, and output the visual detection result.

2. The system according to claim 1, characterized in that, The base model is trained based on sample images collected from all visual detection scenarios, while the visual processing model is trained based on sample images collected from fixed detection scenarios.

3. The system according to claim 1, characterized in that, The basic model has the same model structure as the visual processing model, and the server is specifically used for: Based on the correspondence between the model structure of the base model and the model structure of the visual processing model, the model parameters of the trained base model are transferred to the visual processing model to obtain the visual processing model.

4. The system according to claim 1, characterized in that, The visual processing model is obtained by model distillation of the base model.

5. The system according to claim 1, characterized in that, The server is also used for: Obtain difficult samples from the edge end during visual inspection; The base model is incrementally trained using the difficult samples and their corresponding result labels to obtain the base model after incremental training.

6. The system according to claim 1, characterized in that, The server is also used for: For the aforementioned visual processing model, obtain the difficult samples collected during the visual detection process at the edge of the visual processing model. The basic model is then trained using the difficult samples corresponding to the visual processing model and the result labels corresponding to the difficult samples to obtain the basic model corresponding to the visual processing model.

7. The system according to claim 1, characterized in that, The edge end is also used for: From the image to be detected acquired by itself, identify difficult samples; and / or, acquire difficult samples identified by the user from the image to be detected acquired from the edge.

8. The system according to claim 1, characterized in that, The server is also used for: When the number of difficult samples reaches a preset number, and / or, every preset time interval, incremental training and / or personalized training are performed on the base model using the difficult samples and the result labels corresponding to the difficult samples.

9. The system according to any one of claims 1 to 8, characterized in that, The edge end is also used for: Identify difficult samples from the images to be detected collected by the user. The model parameters of the deployed visual processing model are updated online using the difficult samples and the corresponding result labels, resulting in an updated visual processing model.

10. The system according to claim 9, characterized in that, Specifically, the edge terminal is used to update the model parameters of its deployed visual processing model online using any of the following methods: From the unquantized model parameters included in the visual processing model, model parameters for visual detection of the target object are determined as model parameters to be adjusted; the difficult sample is input into the visual processing model so that the visual processing model performs visual detection of the difficult sample based on the current model parameters and outputs a prediction result; Based on the difference between the prediction result and the result label corresponding to the difficult sample, the parameters of the model to be adjusted are adjusted. Extract the sample features of the difficult samples, and construct a registration sample by combining the sample features and the corresponding result labels; import the registration sample into the visual processing model so that the visual processing model can learn the correspondence between the sample features and the corresponding result labels through a meta-learning mechanism; Extract the histogram features of the difficult samples, and import the histogram features and the corresponding result labels into the visual processing model so that the visual processing model can fuse the histogram features corresponding to the result labels with the deep learning features corresponding to the result labels that have been learned.

11. The system according to any one of claims 1 to 8, characterized in that, The server is a local server deployed locally for the visual inspection scenario.

12. A visual inspection method, characterized in that, An edge-end application is used in a visual inspection system, which further includes a server. The server deploys a base model with visual inspection capabilities, and the edge-end deploys a visual processing model. The visual processing model is obtained by the server through model parameter migration of the base model. The method includes: Acquire the image of the target object to be detected; The image to be detected is input into the visual processing model deployed on its own, and the visual detection result is output.

13. A visual inspection device, characterized in that, An edge device is used in a visual inspection system, the visual inspection system further includes a server, the server deploys a base model, the base model has visual inspection capabilities, and the edge device deploys a visual processing model, the visual processing model is obtained by the server through model parameter migration of the base model, the device includes: The image acquisition module is used to acquire the image of the target object to be detected; The result acquisition module is used to input the image to be detected into its own deployed visual processing model and output the visual detection result.

14. An edge end, characterized in that, A visual processing model is deployed at the edge, and a server is deployed within the visual detection system where the edge is located. A base model is deployed on the server, and the base model possesses visual detection capabilities. The visual processing model is obtained by the server through model parameter transfer of the base model, wherein: The edge end is used to perform the steps of the method described in claim 12.

15. A server, characterized in that, The server is deployed with a base model, which has visual inspection capabilities, and the visual inspection system on which the server is located also includes at least one edge terminal as described in claim 14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in claim 12.

Images