Image analysis method and device based on deep learning, equipment and storage medium

Abstract

The invention relates to an image analysis method and device based on deep learning, equipment and a storage medium, and belongs to the technical field of image analysis, and the method comprises the steps: obtaining a first image based on a historical image database, and carrying out the preprocessing of the first image, and obtaining a data enhancement image; based on the data enhancement image, training the randomly initialized teacher network model to generate a target teacher network model; the target teacher network model is used to guide the randomly initialized student network model to train so as to generate a target student network model, and the guide dimensions at least comprise trunk feature guide, fusion feature guide and detection network feature guide; and inputting a second image into the target student network model to obtain an image analysis result, the second image being a real-time image. According to the invention, the image detection precision is improved.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and in particular to an image analysis method, apparatus, device and storage medium based on deep learning. Background Art

[0002] With the rapid development of computer vision technology, baggage inspection based on X-ray imaging has been widely used in the public transportation industry; the current contraband detection process still requires a lot of manual intervention, and skilled technicians need to view X-ray images to determine the prohibited items in the luggage. However, during the manual screening of prohibited items, the detection of prohibited items may be incorrect due to reasons such as technician fatigue. Therefore, visual technicians are committed to developing intelligent contraband detection systems to overcome these problems. At present, the mainstream technology in the field of intelligent detection is deep learning technology, which is mainly divided into one-stage algorithms and two-stage algorithms. For example, FasteR-CNN, Faster R-CN and RetinaNet are all based on two-stage detection algorithms, while yolov4, yolov5 and yolov7 are all based on one-stage detection algorithms. These detection methods require the use of a large amount of labeled data for supervised training, and by building a deep neural network model, the characteristics of prohibited items can be automatically identified.

[0003] Although these methods have achieved intelligent detection to a certain extent, they still have some shortcomings:

[0004] 1. The accuracy of contraband detection of large models is often much higher than that of small models, but the number of model parameters of large models is also much higher than that of small models, which leads to a poor balance between the detection performance and reasoning speed of the model, and a trade-off must be made;

[0005] 2. The unique feature information of contraband in X-ray images is often not well utilized in model construction, resulting in low detection accuracy in the task of contraband detection in X-ray images.

[0006] Therefore, there is an urgent need to propose an image analysis method, device, equipment and storage medium based on deep learning that can improve detection accuracy. Summary of the invention

[0007] Based on this, it is necessary to provide an image analysis method, device, equipment and storage medium based on deep learning that can improve detection accuracy in response to the above technical problems.

[0008] In a first aspect, a deep learning-based image analysis method is provided, the method comprising:

[0009] Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image;

[0010] Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0011] The target teacher network model is used to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0012] The second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0013] Optionally, based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model includes:

[0014] Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information;

[0015] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map;

[0016] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0017] Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss;

[0018] Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

[0019] Optionally, the calculation method of the first target loss includes:

[0020]

[0021] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

[0022] Optionally, using the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model includes:

[0023] Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps;

[0024] Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure;

[0025] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information;

[0026] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0027] Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained.

[0028] Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

[0029] Optionally, the method for obtaining the second target loss includes:

[0030] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0031] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0032] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0033] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0034] Optionally, the method further includes:

[0035] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0036] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0037]

[0038] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0039] Optionally, inputting a second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image and includes:

[0040] Acquire a second image, and preprocess the second image to obtain a data enhanced image;

[0041] Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information;

[0042] Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

[0043] In a second aspect, a deep learning-based image analysis device is provided, the device comprising:

[0044] A preprocessing module, used to obtain a first image based on a historical image database, and preprocess the first image to obtain a data enhanced image;

[0045] A first training module is used to train the randomly initialized teacher network model based on the data augmented image to generate a target teacher network model;

[0046] A second training module is used to use the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance and detection network feature guidance;

[0047] The detection module is used to input the second image into the target student network model to obtain an image analysis result, and the second image is a real-time image.

[0048] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0049] Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image;

[0050] Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0051] The target teacher network model is used to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0052] The second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0053] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0054] Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image;

[0055] Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0056] The target teacher network model is used to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0057] The second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0058] In a fifth aspect, a computer program product is provided, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0059] Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image;

[0060] Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0061] The target teacher network model is used to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0062] The second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0063] The above-mentioned deep learning-based image analysis method, device, equipment and storage medium, the method includes: based on a historical image database, obtaining a first image, preprocessing the first image to obtain a data enhanced image; based on the data enhanced image, training the randomly initialized teacher network model to generate a target teacher network model; using the target teacher network model to guide the randomly initialized student network model to train to generate a target student network model, wherein the guidance dimension includes at least backbone feature guidance, fusion feature guidance and detection network feature guidance; inputting a second image into the target student network model to obtain an image analysis result, and the second image is a real-time image. The present application directly guides the deep semantic feature information of the student network model at different stages and scales through the teacher network model, increases the gradient of the reverse update, and improves the student network model's ability to extract features, thereby improving the positioning performance and classification accuracy of image detection analysis, ensuring the real-time nature of the algorithm, and thereby improving the accuracy of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A diagram of an application environment of an image analysis method based on deep learning in one embodiment;

[0065] Figure 2 is a flowchart of an image analysis method based on deep learning in one embodiment;

[0066] Figure 3 A schematic diagram of a teacher network model training process of an image analysis method based on deep learning in one embodiment;

[0067] Figure 4 A schematic diagram of a student network model training process of an image analysis method based on deep learning in one embodiment;

[0068] Figure 5 A schematic diagram of weighted convolution of feature channels of an image analysis method based on deep learning in one embodiment;

[0069] Figure 6 A schematic diagram of a real-time image detection process of an image analysis method based on deep learning in one embodiment;

[0070] Figure 7 is a structural block diagram of an image analysis device based on deep learning in one embodiment;

[0071] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like throughout the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as including but not limited to.

[0074] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0075] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0076] like Figure 1 , which is a schematic diagram of a scene of a deep learning-based image analysis system provided in an embodiment of the present application. The deep learning-based image analysis system may include a computer device 100, in which a deep learning-based image analysis device is integrated, such as Figure 1 Computer equipment in.

[0077] In the embodiment of the present application, the computer device 100 is mainly used for: acquiring a first image based on a historical image database, preprocessing the first image to obtain a data enhanced image; training a randomly initialized teacher network model based on the data enhanced image to generate a target teacher network model; using the target teacher network model to guide the training of a randomly initialized student network model to generate a target student network model, wherein the guidance dimension includes at least backbone feature guidance, fusion feature guidance and detection network feature guidance; inputting a second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0078] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0079] It is understandable that the computer device 100 in the embodiment of the present application may also be a terminal, and the terminal may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link, and such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific terminal may be a desktop terminal or a mobile terminal, and the terminal may also be one of a tablet computer, a laptop computer, etc.

[0080] In an embodiment of the present application, the computer device 100 may be a computer device that is communicatively connected to one or more X-ray machines (also known as security inspection machines), and images scanned by one or more X-ray machines may be input into the computer device 100 as images to be detected for image detection and analysis.

[0081] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown. It can be understood that the deep learning-based image analysis system may also include one or more other services, which are not specifically limited here.

[0082] In addition, if Figure 1As shown, the deep learning-based image analysis system may also include a memory 200 for storing historical image sets to generate a historical database. The historical image sets may include image information related to contraband scanned by an X-ray machine, etc.

[0083] It should be noted that Figure 1 The scene diagram of the deep learning-based image analysis system shown is merely an example. The deep learning-based image analysis system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the deep learning-based image analysis system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0084] In one embodiment, Figure 2 As shown, it is a schematic diagram of an embodiment of the image analysis method based on deep learning in the embodiment of the present application. The image analysis method based on deep learning includes steps S1 to S5, which are as follows:

[0085] S1: Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image.

[0086] It should be noted that the first image refers to an X-ray luggage bag image in a historical image database, that is, an original X-ray contraband image, and the historical image database may include multiple X-ray luggage bag images.

[0087] In some specific embodiments, the image preprocessing method includes:

[0088] Read the original X-ray contraband image, denoted as X t ;

[0089] The original X-ray contraband image is subjected to random scaling, random rotation, random folding, mosaic (data enhancement) and other operations to obtain a data enhanced image X'.

[0090] S2: Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model.

[0091] It should be noted that random initialization refers to the random generation of initial values ​​of network model parameters; the process of training the teacher network model after random initialization is the process of adjusting the initial values ​​of the network model parameters; the teacher network model includes at least a deep residual convolution module, a downsampling convolution module, a path fusion module and a detection head network.

[0092] In some specific embodiments, Figure 3 As shown, based on the data augmented image, the randomly initialized teacher network model is trained to generate a target teacher network model including:

[0093] Based on the deep residual convolution module, feature information of the data-enhanced image is extracted, and the feature information includes at least fine-grained feature information and coarse-grained feature information. Specifically, the data-enhanced image X' after data enhancement is input into the teacher network model after random initialization, and the fine-grained feature information and coarse-grained feature information of the data-enhanced image X' are extracted from shallow to deep through the deep residual convolution module, wherein the coarse-grained feature information includes information such as color, texture, edge and corner, and the fine-grained feature information includes more abstract information, that is, deep semantic information;

[0094] Based on the downsampling convolution module, the feature information is downsampled to generate a multi-scale semantic feature map. Specifically, the downsampling convolution module is responsible for downsampling the feature information extracted by the deep residual convolution module to provide feature information of different scales for the deep residual convolution module to learn, and finally generate semantic feature maps of shallow (large), medium and deep (small) scales.

[0095] The multi-scale semantic feature map is subjected to path fusion to obtain a fused semantic feature map. Specifically, a path fusion module is used to perform path fusion on the multi-scale semantic feature map. Get the shallow semantic feature map after fusion Mid-level semantic features and deep semantic features

[0096] Based on the detection head network, the fused semantic feature map is classified and positioned to obtain a first prediction result. At the same time, the first target loss is calculated. Specifically, the semantic features of large, medium and small scales are classified and positioned to obtain the final prediction result through the detection head network. At the same time, the classification loss is calculated and positioning loss Among them, the classification loss is calculated by the cross entropy loss function, and the positioning loss is calculated by the CIOU loss function. Based on this, the calculation formula of the first target loss is as follows:

[0097]

[0098] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss;

[0099] Based on the first prediction result and the first target loss, reverse training is performed to update the teacher network model to obtain the weight of the teacher network model (i.e., the trained teacher network model parameters are obtained) to generate a target teacher network model, wherein the standard for completing the network model training may be that the prediction accuracy reaches a preset value or the loss value reaches a preset standard, and both the preset value and the preset standard may be set according to actual needs.

[0100] S3: Using the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance and detection network feature guidance.

[0101] It should be noted that the student network model includes at least a shallow residual convolution module, a downsampling convolution module, a path fusion module and a detection head network; as above, the randomly initialized student network model (Random Initialization) refers to the randomly generated initial values ​​of the network model parameters; the process of training the randomly initialized student network model is the process of adjusting the initial values ​​of the network model parameters.

[0102] In some specific embodiments, Figure 4 As shown, using the target teacher network model to guide the randomly initialized student network model to train, so as to generate a target student network model includes:

[0103] The data augmented image is input into the target teacher network model, and multiple feature maps are output. Specifically, the original X-ray contraband image is read and recorded as X t , perform random scaling, random rotation, random folding, mosaic and other operations on it to obtain a data enhanced image X', input the data enhanced image X' after data enhancement processing into the trained teacher network model (i.e., the target teacher network model), and execute the process of step S2 to obtain the semantic feature maps of shallow (large), medium, and deep (small) scales And multi-scale fusion semantic feature map and Then fuse the semantic feature map and Input the detection head network to obtain the feature map and

[0104] The data-enhanced image is input into the student network model after random initialization, and feature information of the data-enhanced image is extracted based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and residual structure. Specifically, the data-enhanced image X' is input into the target teacher network model (large model) and the data-enhanced image X' is input into the student network model (small model) after random initialization, wherein the student network model is composed of a shallow residual convolution module and a downsampling convolution module to form a feature extraction backbone of the network, and the shallow residual convolution module is composed of feature channel weighted convolution and a small amount of residual structure, such as Figure 5 The figure shows a schematic diagram of feature channel weighted convolution. The main function of this module is to use the features of feature channel weighted convolution. While extracting spatial features, it also pays attention to channel feature information, so that the module can extract the semantic features of contraband in X-ray images more comprehensively, accurately and effectively. The feature channel weighted convolution will not increase the number of model parameters in design, but will enhance the feature expression ability of the network, thereby improving the feature extraction ability of the student network model.

[0105] Based on the downsampling convolution module, the feature information is downsampled to generate a multi-scale semantic feature map including spatial and channel information. Specifically, the shallow residual convolution module and the downsampling convolution model operation are performed on the data enhanced image X' from shallow to deep to obtain multi-scale fine-grained features and coarse-grained features containing spatial and channel information, and finally generate three scales of semantic feature maps: shallow (large), medium, and deep (small).

[0106] The multi-scale semantic feature map is subjected to path fusion to obtain a fused semantic feature map. Specifically, the multi-scale semantic feature map is In the input path fusion module, path fusion is performed from top to bottom to obtain the fused shallow semantic feature map Mid-level semantic features and deep semantic features

[0107] Based on the detection head network, the fused semantic feature map is classified and positioned to obtain a second prediction result. At the same time, the feature extraction information of the target teacher network model is used to guide the student network model to obtain a second target loss. Specifically, and Input three independent detection head network modules respectively, perform the final feature modeling, and obtain the feature map and At the same time, the three detection head networks classify and locate the semantic features of large, medium and small scales to obtain the final prediction results. That is, the second prediction result;

[0108] Based on the second prediction result and the second target loss, reverse training is performed to update the student network model to obtain the weight of the student network model (i.e., the parameters of the trained student network model are obtained) to generate a target student network model. Similarly, the standard for completing the network model training can be that the prediction accuracy reaches a preset value or the loss value reaches a preset standard. Both the preset value and the preset standard can be set according to actual needs. At this time, the teacher network model does not need to be trained and updated, and finally the weight of the student network model can be obtained.

[0109] In some specific implementations, the method for obtaining the second target loss includes:

[0110] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0111] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0112] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0113] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0114] In some embodiments, the method further comprises:

[0115] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0116] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0117]

[0118] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0119] Specifically, the feature extraction information of the teacher network model is used to directly guide the student network model. The guidance dimension consists of backbone feature guidance, fusion feature guidance and detection network feature guidance. The backbone feature guidance is composed of shallow feature loss L s , middle-level feature loss L m and deep feature loss L d Composition, fusion feature guidance is the shallow semantic feature loss after fusion Mid-level semantic feature loss and deep semantic feature loss The detection network feature guidance is composed of feature losses between large, medium and small detection heads. Composition, yes and The absolute error can be obtained to obtain the backbone feature guidance loss L s , L m , L d ,right and Calculate the absolute error to get the fusion feature guidance loss right and Find the absolute loss to get the detection network guidance loss The final teacher network model guides the knowledge distillation loss of the student network model. KD The formula is as follows:

[0120]

[0121] Prediction results using the student network model and Y gt Calculating classification loss and positioning loss Final Loss s It consists of knowledge distillation loss, classification loss, and positioning loss, and the calculation formula is as follows:

[0122]

[0123] S4: Input the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0124] It should be noted that the second image is an X-ray image of the contraband to be detected acquired in real time.

[0125] In some specific embodiments, Figure 6 As shown, the second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image including:

[0126] Acquire a second image, and preprocess the second image to obtain a data enhanced image, wherein the preprocessing method is the same as the above step, that is, perform random scaling, random rotation, random folding, mosaic and other operations on the second image to obtain a data enhanced image;

[0127] The data-enhanced image is input into the target student network model to obtain a prediction result, which includes location information and classification information. Specifically, the data-enhanced image is input into the trained student network model (i.e., the target student network model), and a shallow residual convolution module and a downsampling convolution module are used to obtain multi-scale fine-grained features and coarse-grained features containing spatial and channel information, i.e., semantic feature maps of shallow (large), medium, and deep (small) scales. Multi-scale semantic features In the input path fusion module, fusion is performed from top to bottom to obtain the fused shallow semantic feature map Mid-level semantic features and deep semantic features Will and Three independent detection head network modules are input respectively. The three detection head networks classify and locate the semantic features of large, medium and small scales, and finally obtain the prediction result Y pre ;

[0128] Based on the prediction result, an image analysis rendering (i.e., a contraband detection rendering) is formed in the data enhanced image to generate the image analysis result. Specifically, according to the predicted position information and classification information, it is drawn in the input image to form a contraband detection rendering.

[0129] In the above-mentioned image analysis method based on deep learning, this application aims at the problems of low accuracy, imbalance in reasoning time and detection accuracy of X-ray image analysis based on deep learning in the field of intelligent security inspection, and proposes an X-ray image analysis method through a multi-scale knowledge distillation strategy, which ultimately realizes the intelligent detection task of contraband with high accuracy and low latency; compared with the current contraband detection algorithm, the X-ray image contraband detection method based on the multi-scale knowledge distillation strategy performs preprocessing through data enhancement to improve the learning ability of the model and the generalization ability of detection; uses a teacher network model with a large number of parameters to guide a student network model with a small number of parameters to learn the feature information of contraband in X-ray images, and adopts a multi-scale strategy to train the student network The deep semantic feature information of the model at different stages and scales is used for direct guidance, so that the student network model retains the key deep semantics of the image in the teacher network model to the maximum extent, increases the gradient of reverse update, and improves the student network's ability to extract features. It not only retains the advantages of high detection accuracy of large models, but also has the advantages of fast reasoning speed of small models, thereby improving the positioning performance and classification accuracy of contraband and ensuring the real-time performance of the algorithm; the feature channel weighted convolution is used to comprehensively, accurately and effectively adaptively learn the channel and spatial information of X-ray images, and without increasing the amount of network parameters, it fully focuses on the feature information of X-ray images, further improves the quality of feature extraction, and thus improves the accuracy of contraband detection.

[0130] It should be understood that although Figure 2-Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 6 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0131] In one embodiment, Figure 7 As shown, a deep learning-based image analysis device is provided, comprising: a preprocessing module, a first training module, a second training module and a detection module, wherein:

[0132] A preprocessing module, used to obtain a first image based on a historical image database, and preprocess the first image to obtain a data enhanced image;

[0133] A first training module is used to train the randomly initialized teacher network model based on the data augmented image to generate a target teacher network model;

[0134] A second training module is used to use the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance and detection network feature guidance;

[0135] The detection module is used to input the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0136] As a preferred implementation, in the embodiment of the present invention, the first training module is specifically used for:

[0137] Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information;

[0138] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map;

[0139] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0140] Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss;

[0141] Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

[0142] As a preferred implementation manner, in the embodiment of the present invention, the first training module is further used for:

[0143]

[0144] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

[0145] As a preferred implementation, in the embodiment of the present invention, the second training module is specifically used for:

[0146] Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps;

[0147] Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure;

[0148] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information;

[0149] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0150] Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained.

[0151] Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

[0152] As a preferred implementation, in the embodiment of the present invention, the second training module is further used for:

[0153] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0154] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0155] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0156] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0157] As a preferred implementation, in the embodiment of the present invention, the second training module is further used for:

[0158] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0159] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0160]

[0161] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0162] As a preferred implementation, in the embodiment of the present invention, the detection module is specifically used for:

[0163] Acquire a second image, and preprocess the second image to obtain a data enhanced image;

[0164] Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information;

[0165] Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

[0166] For the specific definition of the image analysis device based on deep learning, please refer to the definition of the image analysis method based on deep learning above, which will not be repeated here. Each module in the above-mentioned image analysis device based on deep learning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0167] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image analysis method based on deep learning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0168] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0170] S1: Based on a historical image database, obtain a first image, and preprocess the first image to obtain a data enhanced image;

[0171] S2: Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0172] S3: using the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0173] S4: Input the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0174] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0175] Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information;

[0176] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map;

[0177] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0178] Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss;

[0179] Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181]

[0182] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

[0183] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0184] Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps;

[0185] Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure;

[0186] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information;

[0187] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0188] Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained.

[0189] Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

[0190] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0191] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0192] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0193] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0194] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0195] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0196] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0197] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0198]

[0199] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0200] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0201] Acquire a second image, and preprocess the second image to obtain a data enhanced image;

[0202] Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information;

[0203] Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

[0204] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0205] S1: Based on a historical image database, obtain a first image, and preprocess the first image to obtain a data enhanced image;

[0206] S2: Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0207] S3: using the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0208] S4: Input the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0209] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0210] Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information;

[0211] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map;

[0212] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0213] Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss;

[0214] Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0216]

[0217] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0219] Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps;

[0220] Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure;

[0221] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information;

[0222] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0223] Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained.

[0224] Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

[0225] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0226] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0227] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0228] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0229] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0230] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0231] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0232] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0233]

[0234] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0235] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0236] Acquire a second image, and preprocess the second image to obtain a data enhanced image;

[0237] Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information;

[0238] Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

[0239] In one embodiment, a computer program product is provided, the computer program product comprising a computer program, the computer program when executed by a processor implements the following steps:

[0240] S1: Based on a historical image database, obtain a first image, and preprocess the first image to obtain a data enhanced image;

[0241] S2: Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model;

[0242] S3: using the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance;

[0243] S4: Input the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

[0244] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0245] Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information;

[0246] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map;

[0247] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0248] Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss;

[0249] Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

[0250] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0251]

[0252] Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

[0253] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0254] Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps;

[0255] Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure;

[0256] Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information;

[0257] Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map;

[0258] Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained.

[0259] Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

[0260] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0261] Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ;

[0262] Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss

[0263] Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss

[0264] Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

[0265] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0266] Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model;

[0267] Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes:

[0268]

[0269] Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

[0270] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0271] Acquire a second image, and preprocess the second image to obtain a data enhanced image;

[0272] Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information;

[0273] Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

[0274] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0275] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0276] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. An image analysis method based on deep learning, characterized in that: The method comprises: Based on a historical image database, a first image is acquired, and the first image is preprocessed to obtain a data enhanced image; Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model; The target teacher network model is used to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance, and detection network feature guidance; The second image is input into the target student network model to obtain an image analysis result, wherein the second image is a real-time image.

2. The image analysis method based on deep learning according to claim 1, characterized in that: Based on the data augmented image, training the randomly initialized teacher network model to generate a target teacher network model includes: Extracting feature information of the data-enhanced image based on a deep residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information; Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map; Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map; Based on the detection head network, classify and locate the fused semantic feature map to obtain a first prediction result, and at the same time, calculate a first target loss; Based on the first prediction result and the first target loss, reverse training updates the teacher network model to generate a target teacher network model.

3. The image analysis method based on deep learning according to claim 2, characterized in that: The calculation method of the first target loss includes: Among them, Loss t represents the first target loss, represents the true probability distribution of the sample, Indicates the probability distribution of the model's prediction of samples. represents the classification loss, represents the positioning loss.

4. The image analysis method based on deep learning according to claim 1, characterized in that: Using the target teacher network model to guide the randomly initialized student network model to train, so as to generate a target student network model includes: Inputting the data augmented image into the target teacher network model and outputting a plurality of feature maps; Inputting the data-enhanced image into a randomly initialized student network model, extracting feature information of the data-enhanced image based on a shallow residual convolution module, wherein the feature information includes at least fine-grained feature information and coarse-grained feature information, and the shallow residual convolution module is composed of feature channel weighted convolution and a residual structure; Based on the downsampling convolution module, downsampling the feature information to generate a multi-scale semantic feature map including spatial and channel information; Performing path fusion on the multi-scale semantic feature map to obtain a fused semantic feature map; Based on the detection head network, the fused semantic feature map is classified and positioned to predict, and a second prediction result is obtained. Meanwhile, the feature extraction information of the target teacher network model is used to guide the student network model, and a second target loss is obtained. Based on the second prediction result and the second target loss, reverse training updates the student network model to generate a target student network model.

5. The image analysis method based on deep learning according to claim 4, characterized in that: Methods for obtaining the second target loss include: Calculate the absolute error of the semantic feature map of the student network model and the target teacher network model to obtain the backbone feature guidance loss L s , L m , L d ; Calculate the absolute error of the fused feature map of the student network model and the target teacher network model to get the fused feature guidance loss Calculate the absolute error of the detection head network output feature map of the student network model and the target teacher network model to obtain the detection network feature guidance loss Based on the backbone feature guidance loss, the fusion feature guidance loss and the detection network feature guidance loss, a third target loss is calculated.

6. The image analysis method based on deep learning according to claim 5, characterized in that: The method further comprises: Based on the second prediction result of the student network model, determining the classification loss and the positioning loss of the student network model; Based on the classification loss and positioning loss of the student network model and the third target loss, the second target loss is calculated and determined, and the calculation formula includes: Among them, Loss KD represents the third target loss, represents the classification loss of the student network model, Represents the classification loss of the student network model, Loss s represents the second objective loss, and α, β, and γ all represent weights.

7. The image analysis method based on deep learning according to claim 1, characterized in that: Inputting the second image into the target student network model to obtain an image analysis result, wherein the second image is a real-time image including: Acquire a second image, and preprocess the second image to obtain a data enhanced image; Inputting the data augmented image into the target student network model to obtain a prediction result, wherein the prediction result includes position information and classification information; Based on the prediction result, an image analysis effect diagram is formed in the data enhanced image to generate the image analysis result.

8. An image analysis device based on deep learning, characterized in that: The device comprises: A preprocessing module, used to obtain a first image based on a historical image database, and preprocess the first image to obtain a data enhanced image; A first training module is used to train the randomly initialized teacher network model based on the data augmented image to generate a target teacher network model; A second training module is used to use the target teacher network model to guide the randomly initialized student network model for training to generate a target student network model, wherein the guidance dimensions include at least backbone feature guidance, fusion feature guidance and detection network feature guidance; The detection module is used to input the second image into the target student network model to obtain an image analysis result, and the second image is a real-time image.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images