Instrument detection method, system, equipment and medium based on Transformer distillation

Abstract

The present invention relates to an instrument detection method, system, computer equipment and storage medium based on Transformer distillation. The method comprises: performing image preprocessing on an instrument image based on a DCGAN model, and outputting a target instrument image; performing data enhancement processing on the target instrument image through a Diffusion Model network model, and generating training samples with classification features from the target instrument image; performing distillation using Transformer, using a CenterNet network as a teacher network and a CNN network as a student network, and training to obtain a target detection network; performing instrument detection through the target detection network to obtain a detection result. Image denoising and data enhancement based on the DCGAN model and the Diffusion Model network model can reduce the impact of the external environment and random interference of the distribution station on the image; based on Transformer distillation, the model can be lightweight, the detection accuracy can be guaranteed, and the detection accuracy of the target can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an instrument detection method, system, computer equipment and storage medium based on Transformer distillation. Background Art

[0002] With the rapid development of my country's social economy and the rapid advancement of urbanization, the urban distribution network has become a complex aggregate that carries massive operating information and complex power supply equipment. In addition, the construction of the distribution network covers a wide area and a long time span. The complexity of distribution network management has led to the fact that the management methods and efficiency of distribution equipment in some parts of my country cannot meet the power supply companies' demand for high reliability and high-quality services. The distribution station is responsible for the distribution of electric energy in the region, and the stable operation of the basic equipment and components of the power system is the premise for the stable power supply of the distribution station. Usually, there are many instruments and meters such as opening and closing indicators, circuit breakers, discharge counters, and barometers in the distribution station, and they are distributed in every corner of the substation. The inspection personnel need to inspect the equipment of the entire substation in a high-voltage dangerous environment. The labor intensity is high, and it is difficult to ensure the high-quality completion of the inspection task in long-term high-intensity repetitive labor. For this reason, a series of power inspection robots have been launched in China. Substation inspection robots replace manual inspection tasks, and need to realize basic functions such as autonomous navigation and positioning, metering and identification, and automatic charging. Among them, the meter detection and identification function is a very important function of the inspection robot. The robot needs to detect common instruments and automatically read the instrument data, such as ammeters, voltmeters, barometers, etc.

[0003] However, traditional inspection robots use cameras to automatically identify instruments, which is easily affected by factors such as angle and lighting during image acquisition, resulting in unstable image quality. In addition, the types of instrument pointers are diverse, and camera recognition is often inaccurate. Therefore, traditional inspection robots have a low detection accuracy when performing instrument detection. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, an instrument detection method, system, computer equipment and storage medium based on Transformer distillation are provided, which can improve the accuracy of instrument detection.

[0005] An instrument detection method based on Transformer distillation, the method comprising:

[0006] Collecting instrument images, and performing image preprocessing on the instrument images based on the DCGAN model, and outputting target instrument images;

[0007] Performing data enhancement processing on the target instrument image through a Diffusion Model network model, and generating training samples with classification features from the target instrument image;

[0008] Using Transformer for distillation, using the CenterNet network as the teacher network, using the CNN network as the student network, and taking the training samples as input, to train and obtain the target detection network;

[0009] Instrument detection is performed through the target detection network to obtain a detection result.

[0010] In one of the embodiments, the DCGAN model includes a generative network model and a discriminative network model;

[0011] The performing image preprocessing on the instrument image based on the DCGAN model includes:

[0012] The generative network model receives random noise, adds the random noise to the instrument image, generates a false image, and inputs the false image into the discriminative network model;

[0013] Determining the authenticity of the fake image through the discriminant network model and outputting the determination result;

[0014] According to the judgment result, determining whether to use the generative network model to generate an image expansion data set;

[0015] The image expansion data set is used for image denoising.

[0016] In one of the embodiments, the generation network model is a U-Net network structure; the discrimination network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

[0017] In one of the embodiments, the Diffusion Model network model includes a forward process and a backward process;

[0018] The data enhancement processing of the target instrument image by using the Diffusion Model network model includes:

[0019] In the forward process stage, Gaussian noise is continuously added to the target instrument image to obtain an instrument image after adding noise;

[0020] In the backward process stage, the instrument image after adding noise is reversely diffused to restore image data;

[0021] The Diffusion Model network model is trained through the forward process stage and the backward process stage, and data enhancement processing is performed on the target instrument image through the trained Diffusion Model network model.

[0022] In one embodiment, in the backward process stage, the instrument image after adding noise is reversely diffused to restore the image data, including:

[0023] In the backward process stage, the instrument image after adding noise is reversely diffused to obtain a reversed data distribution;

[0024] According to the inverted data distribution, the original image distribution is restored from the standard Gaussian distribution to recover the image data.

[0025] In one embodiment, the method further comprises:

[0026] Obtain a first representation vector of each feature graph in the CenterNet network, and pass the first representation vector into an encoder to obtain a first encoding information matrix;

[0027] Obtain a second representation vector of each feature graph in the CNN network, and pass the second representation vector into an encoder to obtain a second encoding information matrix;

[0028] Information feature interaction is achieved through the first coding information matrix and the second coding information matrix.

[0029] In one embodiment, a Transformer is used for distillation, a CenterNet network is used as a teacher network, a CNN network is used as a student network, the training sample is used as an input, and a target detection network is trained, including:

[0030] The training samples are used as input, and a network model is trained according to the first encoding information matrix and the second encoding information matrix to obtain a target detection network.

[0031] An instrument detection system based on Transformer distillation, the system comprising:

[0032] An image preprocessing module is used to collect instrument images, and perform image preprocessing on the instrument images based on a DCGAN model to output a target instrument image;

[0033] A data enhancement processing module, used for performing data enhancement processing on the target instrument image through a Diffusion Model network model, and generating a training sample with classification features from the target instrument image;

[0034] A Transformer distillation module, for performing distillation using a Transformer, using a CenterNet network as a teacher network, using a CNN network as a student network, and using the training samples as input to train a target detection network;

[0035] The detection module is used to perform instrument detection through the target detection network to obtain a detection result.

[0036] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Collecting instrument images, and performing image preprocessing on the instrument images based on the DCGAN model, and outputting target instrument images;

[0038] Performing data enhancement processing on the target instrument image through a Diffusion Model network model, and generating training samples with classification features from the target instrument image;

[0039] Using Transformer for distillation, using the CenterNet network as the teacher network, using the CNN network as the student network, and taking the training samples as input, to train and obtain the target detection network;

[0040] Instrument detection is performed through the target detection network to obtain a detection result.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0042] Collecting instrument images, and performing image preprocessing on the instrument images based on the DCGAN model, and outputting target instrument images;

[0043] Performing data enhancement processing on the target instrument image through a Diffusion Model network model, and generating training samples with classification features from the target instrument image;

[0044] Using Transformer for distillation, using the CenterNet network as the teacher network, using the CNN network as the student network, and taking the training samples as input, to train and obtain the target detection network;

[0045] Instrument detection is performed through the target detection network to obtain a detection result.

[0046] The above-mentioned instrument detection method, system, computer equipment and storage medium based on Transformer distillation perform image denoising and data enhancement based on the DCGAN model and the Diffusion Model network model, which can reduce the impact of the external environment and random interference of the distribution station on the image and solve the problem of difficulty in manual labeling; based on Transformer distillation, the model can be lightweight, ensuring the detection accuracy while improving the detection accuracy of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is an application environment diagram of an instrument detection method based on Transformer distillation in one embodiment;

[0048] Figure 2 A schematic diagram of a flow chart of an instrument detection method based on Transformer distillation in one embodiment;

[0049] Figure 3 A schematic diagram of a process of performing denoising on an instrument image in one embodiment;

[0050] Figure 4 A hierarchical structure diagram of a discriminant network model in one embodiment;

[0051] Figure 5 A schematic diagram of a backward process of a Diffusion network in an embodiment;

[0052] Figure 6 This is a schematic diagram of a structure based on Transformer distillation in one embodiment;

[0053] Figure 7 A schematic diagram of the system architecture of an instrument detection method based on Transformer distillation in one embodiment;

[0054] Figure 8 It is a structural block diagram of an instrument detection system based on Transformer distillation in one embodiment;

[0055] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe representation vectors, but these representation vectors are not limited by these terms. These terms are only used to distinguish a first representation vector from another representation vector. For example, without departing from the scope of this application, a first representation vector may be referred to as a second representation vector, and similarly, a second representation vector may be referred to as a first representation vector. Both the first representation vector and the second representation vector are representation vectors, but they are not the same representation vector.

[0058] The instrument detection method based on Transformer distillation provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110. The computer device 110 can collect instrument images, and perform image preprocessing on the instrument images based on the DCGAN model, and output a target instrument image; the computer device 110 can perform data enhancement processing on the target instrument image through the Diffusion Model network model, and generate training samples with classification features from the target instrument image; the computer device 110 can use Transformer for distillation, using the CenterNet network as the teacher network, the CNN network as the student network, and the training samples as input to train a target detection network; the computer device 110 can perform instrument detection through the target detection network to obtain a detection result. Among them, the computer device 110 can be, but is not limited to, various personal computers, laptops, robots, unmanned aerial vehicles, tablet computers and other devices.

[0059] In one embodiment, Figure 2 As shown, a method for instrument detection based on Transformer distillation is provided, comprising the following steps:

[0060] Step 202: collect an instrument image, perform image preprocessing on the instrument image based on the DCGAN model, and output a target instrument image.

[0061] The instrument image may be an image containing an instrument pointer. Since the housing of most pointer-type instruments in the distribution station is made of glass, the collected instrument image is easily affected by the light source and angle, which causes fluctuations in image quality and affects the subsequent recognition effect. Therefore, the image needs to be properly preprocessed. Image preprocessing can effectively provide more obvious features for image analysis and recognition. The DCGAN model, i.e., deep convolutional adversarial network, is a deep learning model consisting of two neural networks, a generator and a discriminator. DCGAN combines the ideas of convolutional neural networks and generative adversarial networks. In this embodiment, it can be used to preprocess the meter image to output the target instrument image.

[0062] Step 204: perform data enhancement processing on the target instrument image through the Diffusion Model network model, and generate training samples with classification features from the target instrument image.

[0063] The deep learning model is a data-driven technology that requires sufficient samples to achieve good training results. If small sample data is directly used in deep learning training, overfitting problems will occur. In order to solve the problem of dial labeling and collection difficulties, the Diffusion Model network is used to achieve sample data enhancement. Specifically, the computer device can perform data enhancement processing on the target instrument image through the Diffusion Model network model to generate training samples with classification features from the target instrument image.

[0064] Step 206, using Transformer for distillation, using the CenterNet network as the teacher network, the CNN network as the student network, and the training samples as input to train the target detection network.

[0065] In edge scenarios such as intelligent robot inspection systems, a lightweight target detection algorithm is required to successfully deploy the algorithm on the embedded end. In order to balance detection accuracy and detection speed, this embodiment adopts a model lightweight method based on Transformer distillation.

[0066] This embodiment uses transformer for distillation. As the knowledge sharing module, the teacher network selects CenterNet, while the student network selects a CNN network with a simple structure, a small number of parameters, and a low computational overhead. Compared with training the student model from scratch, the knowledge distillation process has a lower training cost. By inputting training samples, the final target detection network can be trained. Among them, the target detection network can be used for instrument detection.

[0067] Step 208: Perform instrument detection through the target detection network to obtain a detection result.

[0068] After the target detection network is trained, the image of the instrument to be detected can be used as input, and the detection result can be output through the target detection network.

[0069] In this embodiment, image denoising and data enhancement are performed based on the DCGAN model and the Diffusion Model network model, which can reduce the impact of the external environment and random interference of the distribution station on the image and solve the problem of difficulty in manual labeling; based on Transformer distillation, the model can be lightweight, ensuring detection accuracy while improving the detection accuracy of the target.

[0070] In this embodiment, by designing a data set enhancement method for the DCGAN model, the impact of the external environment and random interference of the distribution station on the image is solved and reduced, and the problem of difficulty in manual labeling is solved; on the other hand, by designing an instrument detection and recognition method for the distribution station based on the attention fusion of the improved DCGAN model, the instrument detection task in the intelligent inspection task is solved, while ensuring the detection accuracy, the detection accuracy of the target is improved, which lays a good foundation for the next step of reading the instrument display numbers, and has certain practical value.

[0071] In one embodiment, a provided instrument detection method based on Transformer distillation may also include a process of performing image preprocessing on the instrument image based on the DCGAN model, and the specific process includes: the generative network model receives random noise, adds random noise to the instrument image, generates a fake image, and inputs the fake image into the discriminative network model; judges the authenticity of the fake image through the discriminative network model, and outputs the judgment result; determines whether to use the generative network model to generate an image expansion data set based on the judgment result; wherein the image expansion data set is used for image denoising.

[0072] The noise in the instrument image will interfere with the observed information and reduce the speed of information extraction. Common noises in images include Gaussian, salt and pepper, and uniformly distributed noise. The methods for removing image noise can be divided into mean filtering, median filtering, and Gaussian filtering. Traditionally, filtering has greatly improved the problem of image blur, but too large or too small a smoothing window will have a certain impact on image quality.

[0073] In this embodiment, a denoising method based on GAN is adopted. Different from the traditional denoising method, a two-layer framework is constructed. First, a GAN network is trained with noisy images to learn the noise distribution and generate noise samples, so as to simulate the noisy images in real scenes and solve the problem that the original image lacks corresponding noise; secondly, a CNN network is trained and the sampled noise blocks are used to construct a paired training data set, which is used to train the CNN to denoise a given image.

[0074] In this embodiment, the DCGAN model may include a generative network model and a discriminative network model. The denoising method flow is as follows: Figure 3 As shown, Gaussian noise is added to the instrument image to generate a fake image through the generative network model, and then the fake image is input into the discriminative network model to determine the authenticity of the fake image.

[0075] In one embodiment, the generated network model is a U-Net network structure; the discriminant network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

[0076] Among them, the design structure of the discriminant network model is as follows Figure 4 As shown, Conv represents a 3*3 convolution, a ReLU activation function, and ConvUnit includes a 3*3 convolution, a BN layer, and a ReLU activation unit, which includes a total of 3 layers.

[0077] In one embodiment, a provided instrument detection method based on Transformer distillation may also include a process of performing data enhancement processing on a target instrument image through a Diffusion Model network model, and the specific process includes: in a forward process stage, continuously adding Gaussian noise to the target instrument image to obtain an instrument image after adding noise; in a backward process stage, performing reverse diffusion on the instrument image after adding noise to restore image data; training the Diffusion Model network model through the forward process stage and the backward process stage, and performing data enhancement processing on the target instrument image through the trained Diffusion Model network model.

[0078] Among them, Diffusion Model has certain advantages in terms of stability and quality of generated samples. Diffusion Model is divided into two stages: forward process and backward process. The forward process is to add Gaussian noise through T times of accumulation; the backward process is the process of denoising.

[0079] In the forward process stage, Gaussian noise can be continuously added to the target instrument image to obtain a noisy instrument image.

[0080] Specifically, the forward process is the process of adding noise to the image. Given a real image x0-q(x), the Diffusion forward process adds Gaussian noise to it through T cumulative times to obtain x1, x2, ..., x T . By giving a series of hyperparameters of Gaussian distribution variance Since each moment t in the forward process is only related to the moment t-1, it can also be regarded as a Markov process:

[0081]

[0082] In this Markov process, as t increases, x t is getting closer and closer to pure noise. When T→∞, x T is a completely Gaussian noise, and has the same mean coefficient as And in practice, β t As t increases, it increases, that is, β1<β2<...<β T .

[0083] In one embodiment, a provided instrument detection method based on Transformer distillation may also include a process of restoring image data, and the specific process includes: in the backward process stage, the instrument image after adding noise is reversely diffused to obtain the reversed data distribution; according to the reversed data distribution, the original image distribution is restored from the standard Gaussian distribution to restore the image data.

[0084] In the backward stage, the image data can be restored by performing reverse diffusion on the instrument image after adding noise. Specifically, in this embodiment, the backward process of the Diffusion Model network model is shown as follows: Figure 5 As shown in the figure, the backward stage is to restore the original distribution x0 by gradually reversing it. In this process, each time step is t and t to predict Gaussian noise z θ (x t ,t), then according to Get the mean μ θ (x t ,t), and then get the variance ∑ θ (x t ,t), And think and Finally, the variance can be calculated according to the formula

[0085] Where θ represents the parameter and Gaussian distribution is the noise predicted by the deep model, which can be regarded as z θ (x t ,t), represents the hyperparameter of the 1-Gaussian distribution, represents the average value of the first parameter. Then, using the weight parameter, we can get x t-1 , the loss function of the constructed model is:

[0086]

[0087] in and z θ represents the Gaussian noise to be learned, x0 represents the random noise at the initial moment, Represents the hyperparameters of the 1-Gaussian distribution.

[0088] In one embodiment, a provided instrument detection method based on Transformer distillation may also include: obtaining a first representation vector of each feature map in a CenterNet network, and passing the first representation vector into an encoder to obtain a first encoding information matrix; obtaining a second representation vector of each feature map in a CNN network, and passing the second representation vector into the encoder to obtain a second encoding information matrix; and realizing information feature interaction through the first encoding information matrix and the second encoding information matrix.

[0089] In this embodiment, the representation vector X of each feature map in the CenterNet network is first obtained, and X is obtained by adding the Embedding of the feature map (Embedding is the feature extracted from the original data) and the Embedding of the feature layer position; then, the obtained X representation vector matrix can be passed into the Encoder, and after 6 Encoder blocks, the encoding information matrix C of all words in the sentence can be obtained.

[0090] The encoder part is composed of N encoder layers stacked together. Each encoder layer consists of two sub-layer connection structures. The first sub-layer connection structure includes a multi-head self-attention sub-layer, a normalization layer, and a residual connection. The second sub-layer connection structure includes a feed-forward fully connected sub-layer, a normalization layer, and a residual connection.

[0091] The Decoder part is composed of N stacked Decoder layers, each of which consists of three sub-layer connection structures. The first sub-layer connection structure includes a masked multi-head self-attention sub-layer, a normalization layer, and a residual connection. The second sub-layer connection structure includes a multi-head attention sub-layer (encoder to decoder) and a normalization layer and a residual connection. The third sub-layer connection structure includes a feed-forward fully connected sub-layer, a normalization layer, and a residual connection. The input of the Decoder module is the representation vector Y of each feature map in the student network. Y is obtained by adding the Embedding of the feature map (Embedding is the feature extracted from the original data) and the Embedding of the feature layer position. Then, the feature information interaction is realized through matrix multiplication in the Decoder module.

[0092] In one embodiment, a Transformer distillation-based instrument detection method is provided, which may also include a process of training a target detection network, wherein the specific process includes: taking a training sample as input, training a network model according to a first coding information matrix and a second coding information matrix, and obtaining a target detection network.

[0093] In one embodiment, the CenterNet teacher network structure is:

[0094]

[0095] The overall structure of the CNN student network is:

[0096]

[0097]

[0098] In this embodiment, if Figure 6 As shown in the figure, the feature maps of convolution 2 and convolution 3 in the teacher network and the feature layers of convolution 2 and convolution 3 in the student network can be guided by transformers to achieve parameter sharing between feature maps.

[0099] In one embodiment, a Transformer distillation-based instrument detection method is provided that can be applied to Figure 7 In the system architecture shown, after image preprocessing and data enhancement, knowledge sharing can be achieved through the teacher model and the student model, thereby improving the detection accuracy of instruments in the image.

[0100] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0101] In one embodiment, in order to further test the correctness of the instrument recognition, the improved algorithm is compared with other target detection algorithms, and the test results are as follows:

[0102] Object Detection Algorithms AP50 AP70 Time(ms) YOLOV3 90.84 86.42 64 Faster RCNN 96.34 92.71 210 FCOS 95.12 94.35 46 CenterNet 95.66 93.10 43 Improved student networking 98.23 98.01 47

[0103] It can be seen that the improved student network can correctly identify the instrument with an accuracy of more than 98%.

[0104] In one embodiment, Figure 8As shown, an instrument detection system based on Transformer distillation is provided, including: an image preprocessing module 810, a data enhancement processing module 820, a Transformer distillation module 830 and a detection module 840, wherein:

[0105] The image preprocessing module 810 is used to collect instrument images, perform image preprocessing on the instrument images based on the DCGAN model, and output a target instrument image;

[0106] The data enhancement processing module 820 is used to perform data enhancement processing on the target instrument image through the Diffusion Model network model, and generate a training sample with classification features from the target instrument image;

[0107] Transformer distillation module 830, used to perform distillation using Transformer, using CenterNet network as a teacher network, using CNN network as a student network, using training samples as input, and training to obtain an object detection network;

[0108] The detection module 840 is used to perform instrument detection through a target detection network to obtain a detection result.

[0109] In one embodiment, the DCGAN model includes a generative network model and a discriminative network model; the image preprocessing module 810 is also used to generate a network model to receive random noise, add random noise to the instrument image, generate a false image, and input the false image into the discriminative network model; the authenticity of the false image is judged by the discriminative network model, and the judgment result is output; according to the judgment result, it is determined whether to use the generative network model to generate an image expansion data set; wherein the image expansion data set is used for image denoising.

[0110] In one embodiment, the generated network model is a U-Net network structure; the discriminant network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

[0111] In one embodiment, the Diffusion Model network model includes a forward process and a backward process; the data enhancement processing module 820 is also used to continuously add Gaussian noise to the target instrument image in the forward process stage to obtain the instrument image after adding noise; in the backward process stage, reverse the diffusion of the instrument image after adding noise to restore the image data; the Diffusion Model network model is trained through the forward process stage and the backward process stage, and the data enhancement processing of the target instrument image is performed through the trained Diffusion Model network model.

[0112] In one embodiment, the data enhancement processing module 820 is also used to perform reverse diffusion on the instrument image after adding noise in the backward process stage to obtain the reversed data distribution; according to the reversed data distribution, the original image distribution is restored from the standard Gaussian distribution to restore the image data.

[0113] In one embodiment, the Transformer distillation module 830 is also used to obtain a first representation vector of each feature map in the CenterNet network, and pass the first representation vector into the encoder to obtain a first encoding information matrix; obtain a second representation vector of each feature map in the CNN network, and pass the second representation vector into the encoder to obtain a second encoding information matrix; and realize information feature interaction through the first encoding information matrix and the second encoding information matrix.

[0114] In one embodiment, the detection module 840 is further configured to take the training sample as input, perform network model training according to the first encoding information matrix and the second encoding information matrix, and obtain a target detection network.

[0115] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an instrument detection method based on Transformer distillation is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0116] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0118] Collect instrument images, perform image preprocessing on the instrument images based on the DCGAN model, and output the target instrument images;

[0119] The target instrument image is subjected to data enhancement processing through the Diffusion Model network model, and training samples with classification features are generated from the target instrument image;

[0120] Use Transformer for distillation, use CenterNet as the teacher network, CNN as the student network, and take the training samples as input to train the target detection network.

[0121] Instrument detection is performed through the target detection network to obtain the detection results.

[0122] In one embodiment, the DCGAN model includes a generative network model and a discriminative network model; when the processor executes the computer program, the following steps are also implemented: the generative network model receives random noise, adds random noise to the instrument image, generates a false image, and inputs the false image into the discriminative network model; the authenticity of the false image is judged by the discriminative network model, and the judgment result is output; according to the judgment result, it is determined whether to use the generative network model to generate an image expansion data set; wherein the image expansion data set is used for image denoising.

[0123] In one embodiment, the generated network model is a U-Net network structure; the discriminant network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

[0124] In one embodiment, the Diffusion Model network model includes a forward process and a backward process; when the processor executes the computer program, the following steps are also implemented: in the forward process stage, Gaussian noise is continuously added to the target instrument image to obtain the instrument image after the noise is added; in the backward process stage, the instrument image after the noise is added is reversely diffused to restore the image data; the Diffusion Model network model is trained through the forward process stage and the backward process stage, and the target instrument image is subjected to data enhancement processing through the trained Diffusion Model network model.

[0125] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in the backward process stage, the instrument image after adding noise is reversely diffused to obtain the reversed data distribution; according to the reversed data distribution, the original image distribution is restored from the standard Gaussian distribution to recover the image data.

[0126] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a first representation vector of each feature map in the CenterNet network, and passing the first representation vector into the encoder to obtain a first encoding information matrix; obtaining a second representation vector of each feature map in the CNN network, and passing the second representation vector into the encoder to obtain a second encoding information matrix; and realizing information feature interaction through the first encoding information matrix and the second encoding information matrix.

[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented: taking the training sample as input, training the network model according to the first encoding information matrix and the second encoding information matrix, and obtaining the target detection network.

[0128] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0129] Collect instrument images, perform image preprocessing on the instrument images based on the DCGAN model, and output the target instrument images;

[0130] The target instrument image is subjected to data enhancement processing through the Diffusion Model network model, and training samples with classification features are generated from the target instrument image;

[0131] Use Transformer for distillation, use CenterNet as the teacher network, CNN as the student network, and take the training samples as input to train the target detection network.

[0132] The instrument is detected through the target detection network to obtain the detection results.

[0133] In one embodiment, the DCGAN model includes a generative network model and a discriminative network model; when the computer program is executed by the processor, the following steps are also implemented: the generative network model receives random noise, adds random noise to the instrument image, generates a false image, and inputs the false image into the discriminative network model; the authenticity of the false image is judged by the discriminative network model, and the judgment result is output; according to the judgment result, it is determined whether to use the generative network model to generate an image expansion data set; wherein the image expansion data set is used for image denoising.

[0134] In one embodiment, the generated network model is a U-Net network structure; the discriminant network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

[0135] In one embodiment, the Diffusion Model network model includes a forward process and a backward process; when the computer program is executed by the processor, the following steps are also implemented: in the forward process stage, Gaussian noise is continuously added to the target instrument image to obtain the instrument image after the noise is added; in the backward process stage, the instrument image after the noise is added is reversely diffused to restore the image data; the Diffusion Model network model is trained through the forward process stage and the backward process stage, and the target instrument image is subjected to data enhancement processing through the trained Diffusion Model network model.

[0136] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in the backward process stage, the instrument image after adding noise is reversely diffused to obtain the reversed data distribution; based on the reversed data distribution, the original image distribution is restored from the standard Gaussian distribution to recover the image data.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a first representation vector of each feature map in the CenterNet network, and passing the first representation vector into the encoder to obtain a first encoding information matrix; obtaining a second representation vector of each feature map in the CNN network, and passing the second representation vector into the encoder to obtain a second encoding information matrix; and realizing information feature interaction through the first encoding information matrix and the second encoding information matrix.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: taking the training sample as input, training the network model according to the first encoding information matrix and the second encoding information matrix, and obtaining the target detection network.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0140] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An instrument detection method based on Transformer distillation, characterized in that: The method comprises: Collecting an instrument image, and performing image preprocessing on the instrument image based on a DCGAN model, and outputting a target instrument image; the DCGAN model includes a generative network model and a discriminative network model; performing image preprocessing on the instrument image based on the DCGAN model, including: the generative network model receives random noise, adds the random noise to the instrument image, generates a fake image, and inputs the fake image into the discriminative network model; judging the authenticity of the fake image by the discriminative network model, and outputting a judgment result; determining whether to use the generative network model to generate an image expansion data set according to the judgment result; wherein the image expansion data set is used for image denoising; The target instrument image is subjected to data enhancement processing by means of a Diffusion Model network model, and a training sample with classification features is generated from the target instrument image; the Diffusion Model network model includes a forward process stage and a backward process stage; the data enhancement processing of the target instrument image by means of the Diffusion Model network model includes: in the forward process stage, continuously adding Gaussian noise to the target instrument image to obtain an instrument image after adding noise; in the backward process stage, performing reverse diffusion on the instrument image after adding noise to restore image data; the Diffusion Model network model is trained by means of the forward process stage and the backward process stage, and the target instrument image is subjected to data enhancement processing by means of the trained Diffusion Model network model; Using Transformer for distillation, using the CenterNet network as the teacher network, using the CNN network as the student network, and taking the training samples as input, to train and obtain the target detection network; Instrument detection is performed through the target detection network to obtain a detection result.

2. The instrument detection method based on Transformer distillation according to claim 1, characterized in that: The generation network model is a U-Net network structure; the discrimination network model includes a convolutional layer, a ReLU activation function layer, and a BN layer.

3. The instrument detection method based on Transformer distillation according to claim 1, characterized in that: In the backward process stage, the instrument image after adding noise is reversely diffused to restore image data, including: In the backward process stage, the instrument image after adding noise is reversely diffused to obtain a reversed data distribution; According to the inverted data distribution, the original image distribution is restored from the standard Gaussian distribution to recover the image data.

4. The instrument detection method based on Transformer distillation according to claim 1, characterized in that: The method further comprises: Obtain a first representation vector of each feature graph in the CenterNet network, and pass the first representation vector into an encoder to obtain a first encoding information matrix; Obtain a second representation vector of each feature graph in the CNN network, and pass the second representation vector into an encoder to obtain a second encoding information matrix; Information feature interaction is achieved through the first coding information matrix and the second coding information matrix.

5. The instrument detection method based on Transformer distillation according to claim 4, characterized in that: Transformer is used for distillation, CenterNet network is used as the teacher network, CNN network is used as the student network, the training samples are used as input, and the target detection network is trained, including: The training samples are used as input, and a network model is trained according to the first encoding information matrix and the second encoding information matrix to obtain a target detection network.

6. An instrument detection system based on Transformer distillation, characterized in that: The system comprises: An image preprocessing module is used to collect instrument images, and perform image preprocessing on the instrument images based on a DCGAN model, and output a target instrument image; the DCGAN model includes a generation network model and a discriminant network model; the image preprocessing on the instrument image based on the DCGAN model includes: the generation network model receives random noise, adds the random noise to the instrument image, generates a fake image, and inputs the fake image into the discriminant network model; the authenticity of the fake image is judged by the discriminant network model, and a judgment result is output; according to the judgment result, it is determined whether to use the generation network model to generate an image expansion data set; wherein the image expansion data set is used for image denoising; A data enhancement processing module is used to perform data enhancement processing on the target instrument image through a Diffusion Model network model, and generate training samples with classification features from the target instrument image; the Diffusion Model network model includes a forward process stage and a backward process stage; the data enhancement processing on the target instrument image through the Diffusion Model network model includes: in the forward process stage, continuously adding Gaussian noise to the target instrument image to obtain an instrument image after adding noise; in the backward process stage, performing reverse diffusion on the instrument image after adding noise to restore image data; training the Diffusion Model network model through the forward process stage and the backward process stage, and performing data enhancement processing on the target instrument image through the trained Diffusion Model network model; A Transformer distillation module, for performing distillation using a Transformer, using a CenterNet network as a teacher network, using a CNN network as a student network, and using the training samples as input to train a target detection network; The detection module is used to perform instrument detection through the target detection network to obtain a detection result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images