Target detection network training method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a target detection network training method, apparatus, equipment and storage medium, and relate to the field of image processing technology. The method obtains reference intermediate features and corresponding initial intermediate features, at least inputs the initial intermediate features into the corresponding feature fusion network for feature fusion to obtain fused features, calculates the intermediate loss value corresponding to the corresponding initial residual block based on the fused features and the reference intermediate features, the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of other feature fusion networks is the initial intermediate feature and the fused feature corresponding to the next feature fusion network. At least the initial target detection network is updated based on the intermediate loss value. In the process of teacher-student consistency distillation learning, a feature fusion network is introduced, and aggregation operations are carried out on the multi-level features obtained in the self-training process, thereby realizing cross-level interaction of features, achieving teacher-student consistency regularization, and improving the detection accuracy of the target detection network.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to target detection network training methods, devices, equipment and storage media. Background Art

[0002] The object detection task aims to identify and localize object instances in images. Object detectors are typically trained using source domain data. Once trained, they are then applied to target domain data to perform detection. However, due to the large domain gap between training data and real-world data, the performance of object detectors can be severely degraded in real-world applications.

[0003] The related art introduces the teacher-student consistency distillation method to perform target detection without passive domain adaptation. With the help of the teacher-student mutual learning mechanism, the teacher detector generates pseudo labels with category scores and bounding boxes for the weakly enhanced target domain image, and then inputs the corresponding strongly enhanced image into the student detector, allowing the student detector to train the model based on the supervision signal provided by the pseudo labels. However, in this process, the training process of the student detector lacks effective regularization correction means. The errors caused by the inaccurate pseudo labels generated by the teacher detector will be directly copied to the student detector during the training process, and continuously fed back to the teacher detector during the iterative process, thereby forming an error accumulation phenomenon, which ultimately affects the detection accuracy of the obtained target detector. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a target detection network training method, device, equipment and storage medium to improve the detection accuracy of the target detection network obtained by passive domain adaptation.

[0005] To achieve the above objectives, an initial aspect of an embodiment of the present application proposes a target detection network training method, which is performed using an initial target detection network and a reference target detection network with consistent network architectures, wherein the initial target detection network includes an initial feature extraction network and an initial detector, wherein the initial feature extraction network includes multiple cascaded initial residual blocks, and the reference target detection network includes a reference feature extraction network and a reference detector, wherein the reference feature extraction network includes a reference residual block corresponding to the position of the initial residual block. The method includes:

[0006] The initial image is fed into a reference feature extraction network for feature extraction, and a reference intermediate feature output by each reference residual block is obtained. The enhanced image corresponding to the initial image is fed into an initial feature extraction network for feature extraction, and an initial intermediate feature output by each initial residual block is obtained. The initial intermediate features and the reference intermediate features corresponding to the positions are obtained to obtain a plurality of intermediate feature groups.

[0007] Using the last reference intermediate feature as detection position information, inputting the detection position information into the reference detector to perform target detection to obtain a label result, and inputting the detection position information into the initial detector to perform target detection to obtain a prediction result, calculating a label loss value based on the label result and the prediction result, and obtaining a distillation loss value based on the target detection process;

[0008] For each intermediate feature group, at least the initial intermediate feature is input into the corresponding feature fusion network for feature fusion to obtain a fused feature, and the intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of the other feature fusion networks is the initial intermediate feature and the fused feature corresponding to the next feature fusion network;

[0009] Based on the label loss value, the distillation loss value and at least one of the intermediate loss values, an initial model weight of the initial object detection network is obtained, and a reference model weight of the reference object detection network is generated according to the initial model weight. The iteration is continued until the training is completed to obtain the trained initial object detection network.

[0010] In some embodiments, inputting at least the initial intermediate features into a corresponding feature fusion network to perform feature fusion to obtain fused features includes:

[0011] Obtaining input data corresponding to the feature fusion network;

[0012] If the input data includes the initial intermediate feature and the fused feature corresponding to the next feature fusion network, performing channel feature processing on the next fused feature to obtain a fused channel feature, performing a first convolution operation on the initial intermediate feature to obtain a first intermediate convolution feature, and obtaining the fused feature corresponding to the current feature fusion network based on the fused channel feature and the first intermediate convolution feature;

[0013] If the input data only includes the initial intermediate features, a first convolution operation is performed on the initial intermediate features to obtain second intermediate convolution features, and a second convolution operation is performed on the second intermediate convolution features to obtain the fused features.

[0014] In some embodiments, performing channel feature processing on the next fused feature to obtain a fused channel feature includes:

[0015] Performing a channel averaging operation on the next fusion feature to obtain an average channel fusion feature;

[0016] A bilinear interpolation upsampling operation is performed on the average channel fusion feature to obtain the fused channel feature.

[0017] In some embodiments, obtaining the fused feature corresponding to the current feature fusion network based on the fused channel feature and the first intermediate convolutional feature includes:

[0018] Splicing the fused channel feature and the first intermediate convolution feature in the channel dimension to obtain a channel splicing feature, and performing a first convolution operation on the channel splicing feature to obtain an attention map feature;

[0019] After multiplying the attention map feature with the fused channel feature and the first intermediate convolution feature respectively, the product results are added to obtain an output residual feature;

[0020] A second convolution operation is performed on the output residual feature to obtain the fused feature.

[0021] In some embodiments, calculating the intermediate loss value corresponding to the initial residual block according to the fused feature and the reference intermediate feature includes:

[0022] Obtaining a fused attention map of the fused features and obtaining a reference attention map of the reference intermediate features;

[0023] Obtain the number of network layers of the initial residual block, and for each network layer, obtain the attention mask corresponding to the network layer according to the fused attention map and the reference attention map, and calculate the channel loss value corresponding to the network layer;

[0024] The network layer loss value is obtained by multiplying the channel loss value corresponding to each network layer by the attention mask, and the intermediate loss value is obtained by accumulating the network layer loss values.

[0025] In some embodiments, obtaining the fused attention map of the fused features includes:

[0026] Obtain the channel slice corresponding to each channel of the fusion feature;

[0027] The channel slices are accumulated and averaged to obtain the fused attention map.

[0028] In some embodiments, obtaining the attention mask corresponding to the network layer according to the fused attention map and the reference attention map includes:

[0029] Obtaining height and width parameters of the fusion feature;

[0030] Obtaining the layer fusion graph features and layer reference graph features corresponding to the fused attention graph and the reference attention graph at the network layer;

[0031] Obtaining mask features based on mask hyperparameters, the layer fusion map features, and the layer reference map features;

[0032] After performing a softmax operation on the mask feature, the attention mask is obtained by multiplying it with the height parameter and the width parameter.

[0033] In some embodiments, calculating the channel loss value corresponding to the network layer includes:

[0034] Get at least one channel grouping parameter;

[0035] For each of the channel grouping parameters, performing channel grouping averaging on the layer fusion features to obtain fusion group data, and performing channel grouping averaging on the layer reference intermediate features to obtain reference group data;

[0036] Calculating a mean square error value between the fused grouped data and the reference grouped data corresponding to each of the channel grouping parameters;

[0037] All the mean square error values ​​are accumulated to obtain the channel loss value.

[0038] In some embodiments, obtaining the initial model weight of the initial object detection network based on the label loss value, the distillation loss value, and at least one of the intermediate loss values ​​includes:

[0039] Obtaining a current initial model weight of the initial object detection network, wherein the current initial model weight includes at least a current initial residual weight of each of the initial residual blocks and a current initial detection weight of the initial detector;

[0040] Adjusting the current initial residual weight of the initial residual block at a corresponding position based on the intermediate loss value to obtain an initial residual weight corresponding to the initial residual block;

[0041] Adjusting a current initial detection weight of the initial detector based on the label loss value and the distillation loss value to obtain an initial detection weight of the initial detector;

[0042] The initial model weight is obtained based on the initial residual weight and the initial detection weight.

[0043] In some embodiments, generating reference model weights of the reference object detection network based on the initial model weights includes:

[0044] Obtaining a current reference model weight of the reference object detection network;

[0045] An exponential moving average operation is performed based on a preset decay rate, the current reference model weight, and the initial model weight to obtain the reference model weight.

[0046] In some embodiments, after calculating the intermediate loss value corresponding to the initial residual block according to the fused feature and the reference intermediate feature, the method further includes:

[0047] Obtaining a cosine similarity value between the initial image and the enhanced image;

[0048] If the cosine similarity value is greater than the consistency threshold, calculating an enhanced consistency parameter according to the basic metric value, the cosine similarity value and the consistency threshold; otherwise, using the basic metric value as the enhanced consistency parameter;

[0049] The intermediate loss value is updated using a product of the enhanced consistency parameter and the intermediate loss value.

[0050] In some embodiments, the calculating the enhanced consistency parameter according to the base metric value, the cosine similarity value, and the consistency threshold value includes:

[0051] Calculating a first difference between the cosine similarity value and the consistency threshold, and calculating a second difference between the cosine similarity value and the consistency threshold;

[0052] Using the first difference as a base and the second difference as a power, a middle value is calculated;

[0053] A quotient of the intermediate value and the second difference is calculated, and the enhanced consistency parameter is obtained according to the sum of the quotient and the basic metric value.

[0054] To achieve the above objectives, the reference aspect of the embodiments of the present application proposes a target detection network training device, which is performed using an initial target detection network and a reference target detection network with consistent network architectures, wherein the initial target detection network includes an initial feature extraction network and an initial detector, wherein the initial feature extraction network includes multiple cascaded initial residual blocks, and the reference target detection network includes a reference feature extraction network and a reference detector, wherein the reference feature extraction network includes a reference residual block corresponding to the position of the initial residual block. The device includes:

[0055] Intermediate feature extraction module: used to send the initial image into the reference feature extraction network for feature extraction, obtain the reference intermediate features output by each reference residual block, send the enhanced image corresponding to the initial image into the initial feature extraction network for feature extraction, obtain the initial intermediate features output by each initial residual block, obtain the initial intermediate features and the reference intermediate features corresponding to the position to obtain multiple intermediate feature groups;

[0056] Target detection module: used to use the last reference intermediate feature as detection position information, input the detection position information into the reference detector to perform target detection to obtain a label result, and input the detection position information into the initial detector to perform target detection to obtain a prediction result, and calculate a label loss value based on the label result and the prediction result;

[0057] An intermediate loss calculation module is configured to input at least the initial intermediate feature into the corresponding feature fusion network for feature fusion to obtain a fused feature for each intermediate feature group, and calculate the intermediate loss value corresponding to the corresponding initial residual block based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of the other feature fusion networks are the initial intermediate feature and the fused feature corresponding to the next feature fusion network;

[0058] The model weight update module is used to obtain the initial model weight of the initial target detection network based on the label loss value and at least one of the intermediate loss values, generate the reference model weight of the reference target detection network according to the initial model weight, continue to iterate until the end of training, and obtain the trained initial target detection network.

[0059] To achieve the above objectives, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the above initial aspect when executing the computer program.

[0060] To achieve the above-mentioned purpose, the fourth aspect of the embodiment of the present application proposes a storage medium, which is a storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the above-mentioned initial aspect is implemented.

[0061] The target detection network training method, apparatus, device and storage medium proposed in the embodiments of the present application, wherein the target detection network training method is performed using an initial target detection network and a reference target detection network with consistent network architecture, the initial target detection network includes an initial feature extraction network and an initial detector, the initial feature extraction network includes multiple cascaded initial residual blocks, the reference target detection network includes a reference feature extraction network and a reference detector, and the reference feature extraction network includes a reference residual block corresponding to the position of the initial residual block. Specifically, the initial image is sent to the reference feature extraction network for feature extraction, and the reference intermediate features output by each reference residual block are obtained. The enhanced image corresponding to the initial image is sent to the initial feature extraction network for feature extraction, and the initial intermediate features output by each initial residual block are obtained. The initial intermediate features and reference intermediate features corresponding to the position are obtained to obtain multiple intermediate feature groups. The last reference intermediate feature is then used as the detection position information, the detection position information is input into the reference detector for target detection to obtain a label result, and the detection position information is input into the initial detector for target detection to obtain a prediction result, and the label loss value is calculated based on the label result and the prediction result. Next, for each intermediate feature group, at least the initial intermediate features are input into the corresponding feature fusion network for feature fusion to obtain fused features. The intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused features and the reference intermediate features. The input data of the last feature fusion network is the corresponding initial intermediate features, and the input data of the remaining feature fusion networks is the initial intermediate features and the fused features corresponding to the next feature fusion network. Finally, the initial model weights of the initial object detection network are obtained based on the label loss value, the distillation loss value, and at least one intermediate loss value. The reference model weights of the reference object detection network are generated based on the initial model weights. Iterations are continued until training is completed, resulting in a trained initial object detection network. In this embodiment of the present application, a feature fusion network is introduced into the process of teacher-student consistency distillation learning to perform aggregation operations on the multi-level features obtained during the self-training process, thereby achieving cross-level feature interaction. During the aggregation process, some information lacking in the features of each level can be obtained, thereby making the features ultimately used in the teacher-student consistency distillation learning richer and more comprehensive, thereby achieving teacher-student consistency regularization. In this way, the initial detector can reasonably correct and learn the label results of the reference detector based on the fused features and the consistency constraints between it and the reference detector. In this way, the error caused by a single inaccurate judgment of the reference detector can be reduced, thereby effectively reducing the error accumulation of the labeling results, and ultimately improving the detection accuracy of the obtained target detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the framework of the target detection network training method provided in the embodiment of the present application.

[0063] Figure 2 It is a schematic diagram of the framework of the initial feature extraction network and the reference feature extraction network provided in the embodiment of the present application.

[0064] Figure 3 This is a flowchart of the target detection network training method provided in an embodiment of the present application.

[0065] Figure 4 This is a flowchart provided by an embodiment of the present application of inputting at least initial intermediate features into a corresponding feature fusion network for feature fusion to obtain fused features.

[0066] Figure 5 This is a flowchart of performing channel feature processing on the next fusion feature to obtain a fusion channel feature provided by an embodiment of the present application.

[0067] Figure 6 A flowchart of obtaining the fusion features corresponding to the current feature fusion network based on the fusion channel features and the first intermediate convolution features provided in an embodiment of the present application.

[0068] Figure 7 This is a flowchart of an embodiment of the present application for calculating the intermediate loss value corresponding to the corresponding initial residual block based on the fusion features and the reference intermediate features.

[0069] Figure 8 This is a flowchart of an embodiment of the present application for obtaining an attention mask corresponding to a network layer based on a fused attention map and a reference attention map.

[0070] Figure 9 This is a flowchart of calculating the channel loss value corresponding to the network layer provided in an embodiment of the present application.

[0071] Figure 10 This is a schematic diagram of the intermediate loss value corresponding to the initial residual block provided in an embodiment of the present application.

[0072] Figure 11 This is a flowchart of updating the intermediate loss value provided in an embodiment of the present application.

[0073] Figure 12 Schematic diagram of the change of cosine similarity values ​​at different enhancement levels in the embodiment of the present application.

[0074] Figure 13 This is a flowchart of an embodiment of the present application for obtaining the initial model weights of an initial target detection network based on a label loss value, a distillation loss value, and at least one intermediate loss value.

[0075] Figure 14 This is a structural block diagram of a target detection network training device provided by another embodiment of the present application.

[0076] Figure 15 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0078] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0080] First, let’s analyze some of the terms used in this application:

[0081] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0082] The object detection task aims to identify and localize object instances in images. Object detectors are typically trained using source domain data. Once trained, they are then applied to target domain data to perform detection. However, due to the large domain gap between training data and real-world data, the performance of object detectors can be severely degraded in real-world applications.

[0083] Taking autonomous driving applications as an example, KITTI, Cityscapes, and BDD100K datasets can all serve as source domain data, providing image data for urban traffic scenarios. However, different datasets vary significantly in terms of resolution, lighting conditions, and viewing angle. Furthermore, the training data in these public datasets is mostly collected under clear skies with good visibility, lacking data collected under low-light or inclement weather conditions. While datasets can be expanded by collecting image data covering a wider range of scenarios to mitigate the effects of domain shift during model application, accurately annotating the large amounts of data in these datasets is often extremely costly and time-consuming. Furthermore, access to source domain data for domain adaptation is often strictly restricted due to data transmission, computational resource limitations, and data privacy issues.

[0084] Therefore, a passive domain adaptation target detection process is required. This process adapts the source domain pre-trained detector to the target domain using only unlabeled target domain data, and no source domain data is accessed during the domain adaptation process. The teacher-student consistency distillation method is introduced in related technologies to perform passive domain adaptation target detection. With the help of the teacher-student mutual learning mechanism, the teacher detector generates pseudo labels with category scores and bounding boxes for the weakly enhanced target domain images, and then inputs the corresponding strongly enhanced images into the student detector. The student detector conducts model training based on the supervision signal provided by the pseudo labels, and then updates the student model parameters to the teacher detector through exponential moving average.

[0085] However, due to the inherent bias between the source domain data and the target data, the pseudo-labels generated by the teacher detector inevitably contain many inaccurate bounding boxes and false positive samples. Therefore, the quality of pseudo-labels needs to be evaluated and improved during the learning process. However, in the absence of a supervised metric, it is difficult to generate high-quality pseudo-labels by searching for the optimal confidence threshold, and most supervised methods still rely on noisy pseudo-labels as the only source of supervision.

[0086] The above situation ultimately leads to the lack of effective regularization correction means in the training process of the student detector. The errors caused by the inaccurate pseudo-labels generated by the teacher detector will be directly copied to the student detector during the training process, and continuously fed back to the teacher detector during the iteration process, thereby forming an error accumulation phenomenon and reducing the detection accuracy of the obtained target detector.

[0087] Based on this, the embodiments of the present application provide a target detection network training method, apparatus, equipment and storage medium. By setting a monotonic alignment loss function for attention alignment, the monotonicity of the attention weight sequence is ensured, which helps to achieve rapid convergence of the model. At the same time, it can improve the accuracy of target detection network training, avoid the problem of repetition or omission of words caused by inaccurate speech synthesis results, and improve the naturalness and robustness of the synthesized language.

[0088] The embodiments of the present application provide a target detection network training method, apparatus, device, and storage medium, which are specifically illustrated by the following embodiments. First, the target detection network training method in the embodiments of the present application is described.

[0089] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0090] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0091] The target detection network training method provided in the embodiment of the present application relates to the field of image processing technology. The target detection network training method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be a computer program running in a terminal or a server side. For example, a computer program can be a native program or software module in an operating system; it can be a local (Native) application (Application, APP), that is, a program that needs to be installed in the operating system to run, such as a client that supports target detection network training, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server via a network. The target detection network training method can be executed by a terminal or a server, or by a terminal and a server in collaboration.

[0092] In some embodiments, the terminal can be a smart phone, tablet computer, laptop computer, desktop computer or smart watch, etc. In addition, the terminal can also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the target detection network training method of this embodiment to provide related services to enhance the driving experience. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, and each service node in the blockchain system forms a peer-to-peer (Peer To Peer, P2P) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (Transmission Control Protocol, TCP) protocol. The terminal and the server can be connected via Bluetooth, Universal Serial Bus (USB) or network communication connection methods, and this embodiment is not limited here.

[0093] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0094] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0095] First, the framework of the target detection network training method provided in the embodiment of the present application is described.

[0096] Reference Figure 1 , Figure 1 It is a schematic diagram of the framework of the target detection network training method provided in the embodiment of the present application.

[0097] exist Figure 1 The framework includes an initial target detection network and a reference target detection network, wherein the initial target detection network serves as a student detector and the reference target detection network serves as a teacher detector. The network architectures of the initial target detection network and the reference target detection network are consistent. The initial target detection network includes an initial feature extraction network and an initial detector. The initial feature extraction network includes multiple cascaded initial residual blocks, while the reference target detection network includes a reference feature extraction network and a reference detector. The reference feature extraction network includes a reference residual block corresponding to the position of the initial residual block.

[0098] In one embodiment, in the initial stage of training, the Faster-RCNN detection network is pre-trained using source domain data and serves as both the initial target detection network and the reference target detection network. The source domain data is typically training data related to the target task. The purpose of pre-training is to allow the target detection network to learn some common features and patterns, so that it can converge faster and achieve better performance in subsequent training.

[0099] In one embodiment, referring to Figure 2 , Figure 2 It is a schematic diagram of the framework of the initial feature extraction network and the reference feature extraction network provided in the embodiment of the present application.

[0100] Figure 2 In this paper, both the initial feature extraction network and the reference feature extraction network are based on the ResNet50 framework. As a deep convolutional neural network, ResNet50 includes multiple residual blocks, which effectively address the vanishing gradient problem that occurs during deep network training. The initial feature extraction network consists of multiple cascaded initial residual blocks, four of which are designated as res2, res3, res4, and res5. These initial residual blocks can extract features at different levels, and the corresponding feature levels have different semantic and spatial information from low to high.

[0101] Specifically, res2 is a relatively early feature extraction stage, primarily responsible for extracting basic image features, such as simple edges, fine textures, and other local details. These features contain relatively little semantic information, but they retain a significant amount of spatial information, reflecting the original local structure of the image. res3 is an intermediate stage. Building on the features extracted in res2, it begins integrating local features, developing towards a more semantically robust state. This may reveal local block-like structural features of some objects, representing a key step in the transition from basic features to higher-level features. Compared to res2, the semantic information of these features is richer, while the spatial information is relatively reduced. res4 further strengthens the semantic representation of features. At this point, the extracted features are close to a level of abstraction sufficient to clearly distinguish between different object categories, potentially revealing relatively complete features such as the object's shape and structure. The semantic information of features at this stage is even richer, while the spatial information is further reduced. res5 is a relatively late stage in the network. Its residual blocks ultimately process the features to a very high level of abstraction, making the output features well-suited for final classification and detection tasks, clearly reflecting the key differences between different object categories. It contains extremely rich semantic information, but less spatial information.

[0102] Similarly, refer to Figure 2As shown in the structure of the initial feature extraction network, the reference feature extraction network also includes 4 reference residual blocks corresponding to the positions of the initial residual blocks, which will not be repeated here.

[0103] In one embodiment, in order to carry out aggregation operations on the multi-level features obtained during the self-training process, thereby realizing cross-level interaction of features, refer to Figure 2 , a feature fusion network is added between each relative initial residual block and the reference residual block, and the feature fusion network is used to perform cross-level feature interaction.

[0104] The following combination Figure 1 and Figure 2 Describe the target detection network training method in the embodiment of the present application.

[0105] Figure 3 This is an optional flowchart of the target detection network training method provided in the embodiment of the present application. Figure 3 The method may include but is not limited to steps 110 to 140. It is also understood that this embodiment is Figure 3 The order of step 110 to step 140 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0106] Step 110: Send the initial image to the reference feature extraction network for feature extraction, obtain the reference intermediate features output by each reference residual block, send the enhanced image corresponding to the initial image to the initial feature extraction network for feature extraction, obtain the initial intermediate features output by each initial residual block, obtain the initial intermediate features corresponding to the position and the reference intermediate features to obtain multiple intermediate feature groups.

[0107] In one embodiment, the target domain dataset contains multiple unlabeled initial images. Each initial image is processed using a specific image enhancement algorithm, such as random cropping, color dithering, or Gaussian blurring, to obtain at least one corresponding enhanced image. Each initial image and its corresponding enhanced image are combined into a training sample pair. Following this process, a large number of training sample pairs are systematically generated and subsequently used in subsequent model training, improving the detection performance of the object detection network on the target domain dataset in an unsupervised manner.

[0108] In one embodiment, a training sample pair is selected for the training process. Figure 1 , the initial image belonging to the weak image is sent to the reference feature extraction network of the reference detector for feature extraction, and the reference intermediate features output by each reference residual block are obtained. At the same time, the enhanced image corresponding to the initial image is sent to the initial feature extraction network of the initial detector for feature extraction, and the initial intermediate features output by each initial residual block are obtained. Combined Figure 2 It is found that the initial intermediate feature and the reference intermediate feature corresponding to the position at this time constitute an intermediate feature group. So far, multiple intermediate feature groups can be obtained, such as Figure 2 Four intermediate feature groups can be obtained.

[0109] Step 120: Use the last reference intermediate feature as the detection position information, input the detection position information into the reference detector for target detection to obtain a label result, and input the detection position information into the initial detector for target detection to obtain a prediction result, calculate the label loss value based on the label result and the prediction result, and obtain the distillation loss value based on the target detection process.

[0110] In one embodiment, the reference feature extraction network of the reference object detection network is constructed from a series of cascaded reference residual blocks. In this structure, the reference intermediate features output by the reference residual block at the end of the reference feature extraction network serve as the final output data of the entire reference feature extraction network. Here, the reference intermediate features generated by the last reference residual block are set as the detection position information output by the reference feature extraction network. This detection position information can provide positional data, such as a detection box, corresponding to the target to be detected in the initial image, thereby providing auxiliary information in the subsequent target localization process.

[0111] In one embodiment, referring to Figure 1 In theory, the last initial intermediate feature of the initial feature extraction network of the initial object detection network is also similar to the detection position information. In this case, in order to achieve consistent knowledge distillation between teacher and student, the detection position information output by the reference feature extraction network of the reference object detection network is used to replace the last initial intermediate feature of the initial feature extraction network of the initial object detection network. In other words, the input data of the object detection process of the reference detector and the initial detector is the detection position information output by the reference feature extraction network.

[0112] Therefore, combined Figure 1 , input the detection position information into the reference detector for target detection to obtain the label result. The label result here is the detection result for the target to be detected in the initial image, which can be used as a pseudo label. Then input the detection position information into the initial detector for target detection to obtain the prediction result. Then calculate the label loss value L based on the label result and the prediction result det .

[0113] In addition, the reference detector and the initial detector include multiple network layers. At this time, when the reference detector outputs the label result and the initial detector outputs the prediction result, the feature data corresponding to each network layer is obtained, and the feature loss value of the layer is calculated layer by layer using the mean square error. Then, the feature loss values ​​are summed to obtain the distillation loss value L of the reference detector and the initial detector. KD If the characteristic data of the reference detector and the initial detector differ greatly, the distillation loss value L KD It will be larger.

[0114] Therefore, during the training process, it is necessary to consider the label loss value L det and distillation loss value L KD Perform weight adjustment.

[0115] Next, the process of performing cross-level interaction using the feature fusion module in an embodiment of the present application is described.

[0116] Step 130: For each intermediate feature group, at least the initial intermediate feature is input into the corresponding feature fusion network for feature fusion to obtain a fused feature, and the intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused feature and the reference intermediate feature.

[0117] In one embodiment, combining Figure 2 From the above, we can see that the input data of the last feature fusion network is the corresponding initial intermediate feature, while the input data of other feature fusion networks are the initial intermediate feature and the fusion feature corresponding to the next feature fusion network.

[0118] Specifically, for a feature fusion module at a certain level, its corresponding input is the initial intermediate features of the current level and the fused features output from the higher-level feature fusion network. The fused features output by the feature fusion module at the current level are used to calculate the intermediate loss value of the initial residual block at the current level together with the reference intermediate features of the current level. On the other hand, the fused features output by the current level will continue to be input into the feature fusion module at the previous level. There is a special case here, namely the last feature fusion module. Since it is not connected to a higher-level feature fusion module, its input is the initial intermediate features corresponding to the last level.

[0119] In one embodiment, referring to Figure 4 , Figure 4 This is a flowchart of an embodiment of the present application, which provides at least inputting initial intermediate features into a corresponding feature fusion network for feature fusion to obtain fused features, specifically including the following steps:

[0120] Step 410: Obtain input data corresponding to the feature fusion network.

[0121] In one embodiment, as described above, the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of other feature fusion networks are the initial intermediate feature and the fused feature corresponding to the next feature fusion network.

[0122] Step 420: If the input data includes the initial intermediate feature and the fused feature corresponding to the next feature fusion network, channel feature processing is performed on the next fused feature to obtain a fused channel feature, a first convolution operation is performed on the initial intermediate feature to obtain a first intermediate convolution feature, and a fusion feature corresponding to the current feature fusion network is obtained based on the fused channel feature and the first intermediate convolution feature.

[0123] In one embodiment, referring to Figure 2 Taking the four initial residual blocks as an example, for the fusion feature network corresponding to the first three initial residual blocks, its input data includes the initial intermediate features and the fusion features corresponding to the next feature fusion network. At this time, operations are performed on the initial intermediate features and the fusion features corresponding to the next feature fusion network respectively.

[0124] In one embodiment, referring to Figure 5 , Figure 5 This is a flow chart of performing channel feature processing on the next fusion feature to obtain a fusion channel feature, provided by an embodiment of the present application, which specifically includes the following steps:

[0125] Step 510: Perform channel averaging operation on the next fusion feature to obtain an average channel fusion feature.

[0126] In one embodiment, the channel grouping averaging operation is a computational process that expands processing on the channel dimension. For example, assuming that the number of channels of the next input fusion feature is C', by evenly dividing these channels into several groups (for example, two groups), and then averaging the channel values ​​in each group, the number of channels of the final average channel fusion feature becomes C' / 2. In this embodiment, the channel averaging operation is used to extract more representative channel features while retaining the main information of the features, thereby compressing and simplifying the next fusion feature, reducing redundant information, and also reducing the complexity of subsequent calculations.

[0127] Step 520: Perform a bilinear interpolation upsampling operation on the average channel fusion feature to obtain a fusion channel feature.

[0128] In one embodiment, the width and height of the feature map determine the richness of the spatial information it contains, so it is necessary to perform a bilinear interpolation operation on the average channel fusion feature to increase the size of the feature map.

[0129] Specifically, a weighted average calculation is performed based on the pixel values ​​surrounding the pixel in the average channel fusion feature using preset parameters to determine the value of each pixel in the newly increased size of the fused channel feature. This method expands the feature map in a relatively smooth manner, doubling the width and height of the fused channel feature, allowing it to contain more detailed spatial information, thereby improving the accuracy of subsequent object detection.

[0130] Next, in one embodiment, it is necessary to perform a first convolution operation on the initial intermediate feature to obtain a first intermediate convolution feature. The first convolution operation mentioned here refers to the use of a 1×1 convolution kernel to perform a convolution operation. Although the convolution kernel only covers one pixel in the spatial dimension, this convolution operation acts on all channels of the initial intermediate feature, and uses the 1×1 convolution operation to achieve cross-channel feature interaction, so that the information between different channels can be fused and recombined, thereby enhancing the expression ability of the obtained first intermediate convolution feature.

[0131] In one embodiment, referring to Figure 6 , Figure 6 The flowchart of obtaining the fusion feature corresponding to the current feature fusion network based on the fusion channel feature and the first intermediate convolution feature provided in the embodiment of the present application specifically includes the following steps:

[0132] Step 610: Concatenate the fused channel features and the first intermediate convolutional features in the channel dimension to obtain a channel concatenated feature, and perform a first convolution operation on the channel concatenated feature to obtain an attention map feature.

[0133] In one embodiment, the fused channel feature and the first intermediate convolution feature are spliced ​​in the channel dimension to obtain a channel splicing feature. The so-called splicing in the channel dimension is to connect the two feature maps together in the channel dimension. For example, the size of the fused channel feature is (height, width, C1), where height and width represent length and width respectively, and C1 represents the corresponding number of channels. The size of the first intermediate convolution feature is (height, width, C2), where C2 represents the corresponding number of channels. Then, after splicing in the channel dimension, the size of the channel splicing feature can be expressed as (height, width, C1+C2).

[0134] Next, a 1×1 convolution kernel is used to perform the first convolution operation on the channel splicing features to obtain the attention map features.

[0135] Step 620: After multiplying the attention map feature with the fused channel feature and the first intermediate convolution feature respectively, the product results are added to obtain the output residual feature.

[0136] In one embodiment, the attention map features and the fused channel features are multiplied element-wise according to the multiplication operation rules to obtain a first product matrix. Subsequently, the attention map features and the first intermediate convolution features are multiplied element-wise according to the multiplication operation logic to obtain a second product matrix. Next, the corresponding elements in the first and second product matrices are added one by one to obtain the output residual features. It is understood that the number of channels of the output residual features is 2.

[0137] Step 630: Perform a second convolution operation on the output residual features to obtain fused features.

[0138] In one embodiment, the second convolution operation is to perform a convolution operation using a 3×3 convolution kernel. Therefore, the output residual features are convolved with a 3×3 convolution kernel to further extract local features to obtain fused features.

[0139] Step 430: If the input data only includes the initial intermediate features, a first convolution operation is performed on the initial intermediate features to obtain second intermediate convolution features, and a second convolution operation is performed on the second intermediate convolution features to obtain fused features.

[0140] In one embodiment, referring to Figure 2 Taking four initial residual blocks as an example, the input data for the fusion feature network corresponding to the fourth initial residual block is the fourth initial intermediate feature. A first convolution operation is performed on the initial intermediate feature using a 1×1 convolution kernel to obtain a second intermediate convolution feature. A second convolution operation is then performed on the second intermediate convolution feature using a 3×3 convolution kernel to obtain the fused feature.

[0141] Through the above process, the fusion features corresponding to each intermediate feature group can be obtained. Figure 2 ,Since the intermediate feature group corresponds to the initial residual block, the intermediate loss value corresponding to the initial residual block can be calculated.

[0142] In one embodiment, referring to Figure 7 , Figure 7 This is a flowchart of calculating the intermediate loss value corresponding to the initial residual block according to the fusion feature and the reference intermediate feature provided by the embodiment of the present application, which specifically includes the following steps:

[0143] Step 710: Obtain a fused attention map of the fused features, and obtain a reference attention map of the reference intermediate features.

[0144] In one embodiment, the process of obtaining a fusion attention map of the fusion feature specifically includes: obtaining a channel slice corresponding to each channel of the fusion feature, and then accumulating and averaging the channel slices to obtain a fusion attention map.

[0145] Among them, the fused attention map is expressed as:

[0146]

[0147] Among them, A(Fs) represents the fused attention map, C represents the number of channels of the fused attention map, and F sk；； Represents the channel slice corresponding to the kth channel of the fusion feature. The channel slice is the value of the pixel point corresponding to the fusion feature in this channel.

[0148] The reference attention map of the reference intermediate feature is calculated in the same way as above, where the reference attention map is expressed as:

[0149]

[0150] Among them, A(Ft) represents the reference attention map, C represents the number of channels of the reference attention map, and F tk；； Represents the channel slice corresponding to the kth channel of the reference intermediate feature.

[0151] Step 720: Obtain the number of network layers of the initial residual block, and for each network layer, obtain the attention mask corresponding to the network layer according to the fused attention map and the reference attention map, and calculate the channel loss value corresponding to the network layer.

[0152] In one embodiment, since each initial residual block contains a different number of network layers, it is necessary to determine the number of network layers L corresponding to each initial residual block. During the calculation, the different network layers need to be taken into account. It is understood that the calculation process for the intermediate loss value of each initial residual block is consistent, differing only in the specific data.

[0153] At this point, we randomly select an initial residual block as an example and use the calculation of the lth network layer to illustrate. First, we obtain the lth corresponding attention mask based on the fused attention map and the reference attention map, and then calculate the channel loss value corresponding to the lth network layer.

[0154] In one embodiment, referring to Figure 8 , Figure 8 This is a flowchart of obtaining the attention mask corresponding to the network layer according to the fused attention map and the reference attention map provided by the embodiment of the present application, which specifically includes the following steps:

[0155] Step 810: Obtain the height parameter and width parameter of the fusion feature.

[0156] In one embodiment, the height parameter is the height H of the fused feature, and the width parameter is the width W of the fused feature. It can be understood that the height parameters and width parameters of the fused feature, the fused attention map and the reference attention map are consistent.

[0157] Step 820: Obtain the layer fusion graph features and layer reference graph features corresponding to the fused attention map and the reference attention map at the network layer.

[0158] In one embodiment, the layer fusion graph feature corresponding to the lth network layer of the fused attention graph is represented as The reference attention map corresponding to the layer reference map feature of the lth network layer is represented as

[0159] Step 830: Obtain mask features based on the mask hyperparameters, the layer fusion map features, and the layer reference map features.

[0160] In one embodiment, the mask hyperparameter is τ, which is set according to actual needs. In this case, the mask feature is expressed as:

[0161]

[0162] Step 840: After performing a softmax operation on the mask feature, multiply it by the height parameter and the width parameter to obtain the attention mask.

[0163] In one embodiment, the attention mask corresponding to the lth network layer is expressed as:

[0164]

[0165] in, Denotes the attention mask corresponding to the lth network layer, and σ represents the softmax function. The purpose of multiplying with the height parameter and the width parameter here is to normalize the relevant mask features at each position so that the obtained attention mask is related to the spatial size of the fused attention map and the reference attention map.

[0166] As shown in the calculation process of the attention mask above, when the mask hyperparameter is large, the output of the softmax function is smoother due to the influence of the hyperparameter. This smoothness directly affects the attention mask, making the attention distribution within it highly uniform. In this case, attention is allocated relatively evenly to each location, with no obvious areas of concentrated attention, and the overall attention distribution appears broad and uniform.

[0167] Conversely, when the mask hyperparameter is set to a small value, the output of the softmax function exhibits a completely different characteristic: the output is also smoother, but this smoothness causes the attention distribution in the attention mask to have a sharp shape. This means that attention is highly focused on certain specific locations, while other locations receive relatively less attention, forming a clear distinction between areas of concentrated and non-concentrated attention.

[0168] It can be seen from this that the attention mask in the embodiment of the present application has a powerful representation ability, which can accurately and effectively reflect the consistency of the attention of the fused attention map and the reference attention map at each position in the corresponding network layer.

[0169] Next, calculate the channel loss value corresponding to each network layer.

[0170] In one embodiment, referring to Figure 9 , Figure 9 This is a flowchart of calculating the channel loss value corresponding to the network layer provided in an embodiment of the present application, which specifically includes the following steps:

[0171] Step 910: Obtain at least one channel grouping parameter.

[0172] In one embodiment, referring to Figure 10 , Figure 10 This is a schematic diagram of the intermediate loss value corresponding to the initial residual block provided in the embodiment of the present application. Among them, the channel grouping parameter m can take multiple values, such as Figure 10 In the figure, m = 1, m = 2, and m = 3 are different channel grouping parameters used to determine the number of groups in which the channel dimension is divided. Assuming that the number of channels in the layer fusion feature and the layer reference intermediate feature are both 6, when m = 1, it means that the 6 channels are maintained; when m = 2, it means that the channels are grouped in pairs, resulting in 3 groups; and when m = 3, the channels are divided into groups of three, resulting in 2 groups.

[0173] Step 920: For each channel grouping parameter, perform channel grouping averaging on the layer fusion features to obtain fusion group data, and perform channel grouping averaging on the layer reference intermediate features to obtain reference group data.

[0174] In one embodiment, after the channels are grouped, a channel group averaging operation is performed.

[0175] For layer fusion features, the data within each channel group is averaged. During this process, the system traverses all data elements within each channel group, applies an averaging algorithm, sums these data, and divides them by the number of elements in the group to obtain the average value of each channel group. These average values ​​together constitute the fused group data.

[0176] Similarly, the same channel group averaging process is applied to the layer reference intermediate features. This involves rigorously averaging the data within each channel group, into which the reference intermediate features have been divided. By meticulously summing and averaging the data within each channel group, we obtain the average value for each channel group. These average values ​​are combined to form the reference group data.

[0177] Step 930: Calculate the mean square error between the fused grouped data and the reference grouped data corresponding to each channel grouping parameter.

[0178] In one embodiment, referring to Figure 10 , each channel grouping parameter corresponds to a mean square error value, for example, Figure 10 When m=1, the mean square error value is L1, when m=2, the mean square error value is L2, and when m=3, the mean square error value is L3.

[0179] Specifically, the mean square error value of the i-th channel grouping parameter is expressed as:

[0180]

[0181] in, represents the fused grouping data of the i-th channel grouping parameters, represents the reference grouping data of the i-th channel grouping parameter, and L2() represents the mean square error calculation function.

[0182] Step 940: Accumulate all mean square error values ​​to obtain a channel loss value.

[0183] In one embodiment, 2 l-1 Indicates the number of channel grouping parameters, so the channel loss value is expressed as:

[0184]

[0185] According to the above process, the channel loss value and attention mask corresponding to each network layer are obtained.

[0186] Step 730: Multiply the channel loss value corresponding to each network layer by the attention mask to obtain the network layer loss value, and accumulate the network layer loss values ​​to obtain the intermediate loss value.

[0187] In one embodiment, the channel loss value corresponding to each network layer is multiplied by the attention mask to obtain the network layer loss value corresponding to the network layer, and then all the network layer loss values ​​are accumulated to obtain the intermediate loss value, where the intermediate loss value is expressed as:

[0188]

[0189] Among them, L CAD Represents the intermediate loss value.

[0190] Through the above process, the intermediate loss value corresponding to each initial residual block can be obtained, and the intermediate loss value can reflect the consistency of the local features of the enhanced target domain.

[0191] In one embodiment, since the enhanced image is obtained by performing an enhancement operation on the initial image, if the difference between the enhanced image and the initial image is too large, it will cause a large deviation in the features learned by the reference target detection network and the initial target detection network. For example, when processing the initial data, the reference target detection network learns features that are relatively stable and close to the source domain data. However, when processing the enhanced data, the initial target detection network learns features that are significantly different from the source domain data. Since the data feature distributions processed by the two networks are inconsistent, the initial target detection network cannot effectively perform feature matching and knowledge transfer in the process of distilling knowledge from the reference target detection network, thereby interfering with the final knowledge distillation effect. Therefore, the reference target detection network is used to detect the features of the original image. Figure 1 , the embodiment of the present application also introduces an enhanced consistency parameter to update the intermediate loss value.

[0192] In one embodiment, referring to Figure 11 , Figure 11 This is a flowchart of updating the intermediate loss value provided by an embodiment of the present application. Specifically, it includes the following steps:

[0193] Step 1110: Obtain the cosine similarity value between the initial image and the enhanced image.

[0194] In one embodiment, the corresponding cosine similarity value S is calculated and obtained based on the initial image and the enhanced image. C It should be noted that there is a correlation between the enhancement degree of the enhanced image and the cosine similarity value.

[0195] Reference Figure 12 , Figure 12 The diagram below shows the cosine similarity values ​​for different enhancement levels in the embodiment of the present application. The horizontal axis is the enhancement level, which can be evaluated by the pixel difference between the initial image and the enhanced image, and the vertical axis is the cosine similarity value. Figure 12 As can be seen in the figure, the greater the degree of enhancement, the smaller the cosine similarity value of the original image and the enhanced image, that is, the two are negatively correlated. Based on this, the embodiment of the present application introduces an enhancement consistency parameter to reduce the negative impact caused by the inconsistency between the enhanced image and the original image.

[0196] Step 1120: If the cosine similarity value is greater than the consistency threshold, the enhanced consistency parameter is calculated according to the basic metric value, the cosine similarity value and the consistency threshold; otherwise, the basic metric value is used as the enhanced consistency parameter.

[0197] In one embodiment, a basic metric value β and a consistency threshold α are set according to actual needs. If the cosine similarity value is greater than the consistency threshold, an enhanced consistency parameter is calculated based on the basic metric value, the cosine similarity value and the consistency threshold. The specific calculation process is described as follows: calculating a first difference between the cosine similarity value and the consistency threshold, and calculating a second difference between the cosine similarity value and the consistency threshold, taking the first difference as the base and the second difference as the power to calculate an intermediate value, calculating a quotient of the intermediate value and the second difference, and obtaining the enhanced consistency parameter based on the sum of the quotient and the basic metric value.

[0198] Among them, the enhanced consistency parameter is expressed as:

[0199]

[0200] Among them, S C -α represents the first difference, 1-α represents the second difference, (S C -α) (1-α) represents the middle value, represents the quotient value, W A Indicates enhanced consistency parameters.

[0201] If the cosine similarity value is less than or equal to the consistency threshold, the base metric value is used as the enhanced consistency parameter. In other words, the enhanced consistency parameter is expressed as:

[0202]

[0203] Step 1130: Update the intermediate loss value using the product of the enhanced consistency parameter and the intermediate loss value.

[0204] In one embodiment, combining Figure 1 , according to each training sample pair, an enhanced consistency parameter can be obtained. During calculation, the enhanced consistency parameter is multiplied by each intermediate loss value to update the corresponding intermediate loss value, and a lower enhanced consistency parameter is adaptively assigned to enhanced images with higher inconsistency, reducing their influence on the weights of the two networks during training. Figure 1 , using L TSCD Indicates the updated intermediate loss value.

[0205] Step 140: Based on the label loss value, the distillation loss value and at least one intermediate loss value, an initial model weight of the initial object detection network is obtained. A reference model weight of the reference object detection network is generated according to the initial model weight. The iteration is continued until the training is completed to obtain a trained initial object detection network.

[0206] In one embodiment, after obtaining the label loss value and at least one intermediate loss value, the weight of the initial object detection network needs to be adjusted according to the loss value. Figure 13, Figure 13 This is a flowchart of obtaining the initial model weight of the initial object detection network based on the label loss value, the distillation loss value, and at least one intermediate loss value provided by an embodiment of the present application, specifically including the following steps:

[0207] Step 1310: Obtain the current initial model weights of the initial object detection network.

[0208] In one embodiment, during each iteration, the current initial model weights of the initial target detection network are obtained, wherein the current initial model weights include at least the current initial residual weights of each initial residual block and the current initial detection weights of the initial detector.

[0209] Step 1320: Adjust the current initial residual weight of the initial residual block at the corresponding position based on the intermediate loss value to obtain the initial residual weight corresponding to the initial residual block.

[0210] Step 1330: Adjust the current initial detection weight of the initial detector based on the label loss value to obtain the initial detection weight of the initial detector.

[0211] Step 1340: Obtain initial model weights based on the initial residual weights and the initial detection weights.

[0212] In one embodiment, the current initial residual weights of the initial residual blocks are adjusted based on the intermediate loss values ​​corresponding to each initial residual block. The adjusted weights are defined as the initial residual weights. Simultaneously, the current initial detection weights of the initial detector are adjusted based on the label loss values. The adjusted weights are defined as the initial detection weights. Finally, the initial model weights are derived based on the initial residual weights and the initial detection weights.

[0213] Next, based on the update mechanism of the teacher-student consistency knowledge distillation network, the initial model weights of the updated initial target detection network are used to update the reference model weights corresponding to the reference target detection network using the exponential moving average method. The formula for exponential moving average is:

[0214] θ t =γθ t-1 +(1-γ)θ s

[0215] Among them, θ t represents the current reference model weight, θ t-1 represents the reference model weight in the previous iteration, θ s Represents the current initial model weight, γ represents the decay rate, usually between 0 and 1, and is set according to actual needs.

[0216] From the above process, it can be seen that feature knowledge distillation is introduced in the target detection network training method of the embodiment of the present application as a supplement to pseudo-label supervision. At the same time, a feature fusion network is introduced to perform aggregation operations on the multi-level features obtained during the self-training process, thereby realizing cross-level interaction of features. In the aggregation process, some information that is lacking in the features of each level itself can be obtained, so that the features finally used for teacher-student consistency distillation learning become richer and more comprehensive, thereby achieving teacher-student consistency regularization. In addition, an enhanced consistency parameter is introduced to adaptively assign lower enhanced consistency parameters to enhanced images with higher inconsistency, thereby reducing its influence on the weights of the two networks during the training process.

[0217] The following describes the test results of the target detection network training method according to the embodiment of the present application.

[0218] First, five public datasets were selected during the test, as follows:

[0219] 1) CityScapes dataset: This is a city street scene dataset covering 5,000 finely annotated images.

[0220] 2) FoggyCityScapes dataset: This dataset is a dataset of synthetic fog based on urban landscapes. Each image can provide three fog intensities (0.005, 0.01, and 0.02).

[0221] 3) KITTI dataset: It is a widely used autonomous driving dataset containing 7,481 training images of real street scenes.

[0222] 4) Sim10k dataset: This dataset is a synthetic dataset consisting of 10,000 car and city landscape images generated by the GTA game engine.

[0223] 5) BDD100k dataset: This is a large-scale driving dataset that includes 100,000 images at different times, weather conditions, and driving scenarios.

[0224] For example, we used the CityScapes dataset as the source domain data and the FoggyCityScapes dataset as the target domain data to conduct a training experiment on a non-source domain object detection model. During the experiment, we used the mean average precision (mAP) with an intersection-over-union (IoU) threshold of 0.5 as the evaluation metric.

[0225] It can be seen from the experimental results that the knowledge distillation performance of the target detection network training method provided in the embodiment of the present application is significantly improved by 3.6% mAP compared with the knowledge distillation based on soft labels in the related technology. This proves that the target detection network training method of the embodiment of the present application can effectively improve the accuracy of the final target detection results by providing teacher-student consistency regularization.

[0226] In addition, if only the feature fusion network is added, the performance can be improved by 1.7% mAP to 38.1% mAP compared with the method of the related art. This shows that the target detection network training method of the embodiment of the present application aggregates higher-level features to lower levels. This process can enhance the cross-level interaction ability between the teacher and student networks. If, on the basis of the feature fusion network, the intermediate loss value is further used for training, the performance can be improved by 0.6% mAP compared with the method of the related art. This means that the local feature consistency in the target detection network training method of the embodiment of the present application strengthens the feature representation of the target domain. If, on the basis of the feature fusion network, the intermediate loss value is also updated by using the enhanced consistency parameter, the performance can be improved to 39.4% mAP compared with the method of the related art. This shows that the target detection network training method of the embodiment of the present application can adaptively assign lower enhanced consistency parameters to enhanced images with higher inconsistency, thereby reducing its influence on the weights of the two networks during training.

[0227] The technical solution provided by the embodiment of the present application is executed using an initial target detection network and a reference target detection network with consistent network architecture. The initial target detection network includes an initial feature extraction network and an initial detector. The initial feature extraction network includes multiple cascaded initial residual blocks. The reference target detection network includes a reference feature extraction network and a reference detector. The reference feature extraction network includes a reference residual block corresponding to the position of the initial residual block. Specifically, the initial image is sent to the reference feature extraction network for feature extraction, and the reference intermediate features output by each reference residual block are obtained. The enhanced image corresponding to the initial image is sent to the initial feature extraction network for feature extraction, and the initial intermediate features output by each initial residual block are obtained. The initial intermediate features and reference intermediate features corresponding to the positions are obtained to obtain multiple intermediate feature groups. The last reference intermediate feature is then used as the detection position information, and the detection position information is input into the reference detector for target detection to obtain a label result, and the detection position information is input into the initial detector for target detection to obtain a prediction result. The label loss value is calculated based on the label result and the prediction result. Next, for each intermediate feature group, at least the initial intermediate features are input into the corresponding feature fusion network for feature fusion to obtain fused features. The intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused features and the reference intermediate features. The input data of the last feature fusion network is the corresponding initial intermediate features, and the input data of the remaining feature fusion networks is the initial intermediate features and the fused features corresponding to the next feature fusion network. Finally, the initial model weights of the initial object detection network are obtained based on the label loss value, the distillation loss value, and at least one intermediate loss value. The reference model weights of the reference object detection network are generated based on the initial model weights. Iterations are continued until training is completed, resulting in a trained initial object detection network. In this embodiment of the present application, a feature fusion network is introduced into the process of teacher-student consistency distillation learning to perform aggregation operations on the multi-level features obtained during the self-training process, thereby achieving cross-level feature interaction. During the aggregation process, some information lacking in the features of each level can be obtained, thereby making the features ultimately used in the teacher-student consistency distillation learning richer and more comprehensive, thereby achieving teacher-student consistency regularization. In this way, the initial detector can reasonably correct and learn the label results of the reference detector based on the fused features and the consistency constraints between it and the reference detector. In this way, the error caused by a single inaccurate judgment of the reference detector can be reduced, thereby effectively reducing the error accumulation of the labeling results, and ultimately improving the detection accuracy of the obtained target detection network.

[0228] The present application also provides a target detection network training device, which can implement the target detection network training method described above. Figure 14 , the device comprises:

[0229] Intermediate feature extraction module 1410: used to send the initial image into the reference feature extraction network for feature extraction, obtain the reference intermediate features output by each reference residual block, send the enhanced image corresponding to the initial image into the initial feature extraction network for feature extraction, obtain the initial intermediate features output by each initial residual block, obtain the initial intermediate features corresponding to the position and the reference intermediate features to obtain multiple intermediate feature groups.

[0230] Target detection module 1420: used to use the last reference intermediate feature as detection position information, input the detection position information into the reference detector for target detection to obtain a label result, and input the detection position information into the initial detector for target detection to obtain a prediction result, and calculate the label loss value based on the label result and the prediction result.

[0231] Intermediate loss calculation module 1430: for each intermediate feature group, at least input the initial intermediate feature into the corresponding feature fusion network for feature fusion to obtain a fused feature, and calculate the intermediate loss value corresponding to the corresponding initial residual block based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of other feature fusion networks are the initial intermediate feature and the fused feature corresponding to the next feature fusion network.

[0232] Model weight update module 1440: used to obtain the initial model weight of the initial target detection network based on the label loss value, the distillation loss value and at least one intermediate loss value, generate the reference model weight of the reference target detection network according to the initial model weight, continue to iterate until the end of training, and obtain the trained initial target detection network.

[0233] The specific implementation of the target detection network training device of this embodiment is basically the same as the specific implementation of the above-mentioned target detection network training method, and will not be repeated here.

[0234] An embodiment of the present application further provides an electronic device, including:

[0235] at least one memory;

[0236] at least one processor;

[0237] at least one program;

[0238] The program is stored in the memory, and the processor executes the at least one program to implement the target detection network training method implemented in this application. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0239] See also Figure 15 , Figure 15 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0240] The processor 1501 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0241] The memory 1502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1502 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1502 and is called by the processor 1501 to execute the target detection network training method of the embodiment of the present application;

[0242] Input / output interface 1503, used to implement information input and output;

[0243] Communication interface 1504, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0244] Bus 1505 , which transmits information between various components of the device (e.g., processor 1501 , memory 1502 , input / output interface 1503 , and communication interface 1504 );

[0245] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0246] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned target detection network training method is implemented.

[0247] The memory, as a non-transient storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0248] The target detection network training method, apparatus, device and storage medium proposed in the embodiments of the present application are as follows: the initial image is fed into the reference feature extraction network for feature extraction, the reference intermediate features output by each reference residual block are obtained, the enhanced image corresponding to the initial image is fed into the initial feature extraction network for feature extraction, the initial intermediate features output by each initial residual block are obtained, the initial intermediate features corresponding to the positions and the reference intermediate features are obtained to obtain multiple intermediate feature groups. The last reference intermediate feature is then used as the detection position information, the detection position information is input into the reference detector for target detection to obtain the label result, and the detection position information is input into the initial detector for target detection to obtain the prediction result, and the label loss value is calculated based on the label result and the prediction result. Next, for each intermediate feature group, at least the initial intermediate feature is input into the corresponding feature fusion network for feature fusion to obtain the fused feature, and the intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of the other feature fusion networks are the initial intermediate feature and the fused feature corresponding to the next feature fusion network. Finally, the initial model weights of the initial target detection network are obtained based on the label loss value, the distillation loss value and at least one intermediate loss value, and the reference model weights of the reference target detection network are generated according to the initial model weights. The iteration is continued until the end of the training to obtain the trained initial target detection network. In the process of teacher-student consistency distillation learning, the embodiment of the present application introduces a feature fusion network, and performs aggregation operations on the multi-level features obtained in the self-training process, thereby realizing cross-level interaction of features. In the aggregation process, some information that is lacking in the features of each level itself can be obtained, thereby making the features finally used for teacher-student consistency distillation learning richer and more comprehensive, thereby achieving teacher-student consistency regularization. In this way, the initial detector can reasonably correct and learn the label results of the reference detector based on the fused features and the consistency constraints with the reference detector. In this way, the error caused by a single inaccurate judgment of the reference detector can be reduced, thereby effectively reducing the error accumulation of the label results, and ultimately improving the detection accuracy of the obtained target detection network.

[0249] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0250] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0251] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0252] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0253] The terms "initial," "reference," "third," "fourth," and the like (if any) in the specification of the present application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the numerals used in this manner are interchangeable where appropriate so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or apparatus.

[0254] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0256] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0257] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0258] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0259] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A target detection network training method, characterized in that: The method is performed using an initial target detection network and a reference target detection network having consistent network architectures, wherein the initial target detection network includes an initial feature extraction network and an initial detector, wherein the initial feature extraction network includes a plurality of cascaded initial residual blocks, and the reference target detection network includes a reference feature extraction network and a reference detector, wherein the reference feature extraction network includes a reference residual block corresponding to a position of the initial residual block. The initial image is fed into a reference feature extraction network for feature extraction, and a reference intermediate feature output by each reference residual block is obtained. The enhanced image corresponding to the initial image is fed into an initial feature extraction network for feature extraction, and an initial intermediate feature output by each initial residual block is obtained. The initial intermediate features and the reference intermediate features corresponding to the positions are obtained to obtain a plurality of intermediate feature groups. Using the last reference intermediate feature as detection position information, inputting the detection position information into the reference detector to perform target detection to obtain a label result, and inputting the detection position information into the initial detector to perform target detection to obtain a prediction result, calculating a label loss value based on the label result and the prediction result, and obtaining a distillation loss value based on the target detection process; For each intermediate feature group, at least the initial intermediate feature is input into the corresponding feature fusion network for feature fusion to obtain a fused feature, and the intermediate loss value corresponding to the corresponding initial residual block is calculated based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of the other feature fusion networks is the initial intermediate feature and the fused feature corresponding to the next feature fusion network; Based on the label loss value, the distillation loss value and at least one of the intermediate loss values, an initial model weight of the initial object detection network is obtained, and a reference model weight of the reference object detection network is generated according to the initial model weight. The iteration is continued until the training is completed to obtain the trained initial object detection network.

2. The target detection network training method according to claim 1, characterized in that The step of inputting at least the initial intermediate features into a corresponding feature fusion network to perform feature fusion to obtain fused features includes: Obtaining input data corresponding to the feature fusion network; If the input data includes the initial intermediate feature and the fused feature corresponding to the next feature fusion network, performing channel feature processing on the next fused feature to obtain a fused channel feature, performing a first convolution operation on the initial intermediate feature to obtain a first intermediate convolution feature, and obtaining the fused feature corresponding to the current feature fusion network based on the fused channel feature and the first intermediate convolution feature; If the input data only includes the initial intermediate features, a first convolution operation is performed on the initial intermediate features to obtain second intermediate convolution features, and a second convolution operation is performed on the second intermediate convolution features to obtain the fused features.

3. The target detection network training method according to claim 2, characterized in that: The performing channel feature processing on the next fused feature to obtain a fused channel feature includes: Performing a channel averaging operation on the next fusion feature to obtain an average channel fusion feature; A bilinear interpolation upsampling operation is performed on the average channel fusion feature to obtain the fused channel feature.

4. The target detection network training method according to claim 2, characterized in that The obtaining the fusion feature corresponding to the current feature fusion network based on the fusion channel feature and the first intermediate convolution feature includes: Splicing the fused channel feature and the first intermediate convolution feature in the channel dimension to obtain a channel splicing feature, and performing a first convolution operation on the channel splicing feature to obtain an attention map feature; After multiplying the attention map feature with the fused channel feature and the first intermediate convolution feature respectively, the product results are added to obtain an output residual feature; A second convolution operation is performed on the output residual feature to obtain the fused feature.

5. The target detection network training method according to claim 1, characterized in that The calculating, according to the fusion feature and the reference intermediate feature, an intermediate loss value corresponding to the initial residual block includes: Obtaining a fused attention map of the fused features and obtaining a reference attention map of the reference intermediate features; Obtain the number of network layers of the initial residual block, and for each network layer, obtain the attention mask corresponding to the network layer according to the fused attention map and the reference attention map, and calculate the channel loss value corresponding to the network layer; The network layer loss value is obtained by multiplying the channel loss value corresponding to each network layer by the attention mask, and the intermediate loss value is obtained by accumulating the network layer loss values.

6. The target detection network training method according to claim 5, characterized in that: The obtaining of the fusion attention map of the fusion feature includes: Obtain the channel slice corresponding to each channel of the fusion feature; The channel slices are accumulated and averaged to obtain the fused attention map.

7. The target detection network training method according to claim 5, characterized in that: The obtaining, according to the fused attention map and the reference attention map, an attention mask corresponding to the network layer, comprises: Obtaining height and width parameters of the fusion feature; Obtaining the layer fusion graph features and layer reference graph features corresponding to the fused attention graph and the reference attention graph at the network layer; Obtaining mask features based on mask hyperparameters, the layer fusion map features, and the layer reference map features; After performing a softmax operation on the mask feature, the attention mask is obtained by multiplying it with the height parameter and the width parameter.

8. The target detection network training method according to claim 7, characterized in that: Calculating the channel loss value corresponding to the network layer includes: Get at least one channel grouping parameter; For each of the channel grouping parameters, performing channel grouping averaging on the layer fusion features to obtain fusion group data, and performing channel grouping averaging on the layer reference intermediate features to obtain reference group data; Calculating a mean square error value between the fused grouped data and the reference grouped data corresponding to each of the channel grouping parameters; All the mean square error values ​​are accumulated to obtain the channel loss value.

9. The target detection network training method according to claim 1, characterized in that: Obtaining an initial model weight of the initial object detection network based on the label loss value, the distillation loss value, and at least one of the intermediate loss values ​​includes: Obtaining a current initial model weight of the initial object detection network, wherein the current initial model weight includes at least a current initial residual weight of each of the initial residual blocks and a current initial detection weight of the initial detector; Adjusting the current initial residual weight of the initial residual block at a corresponding position based on the intermediate loss value to obtain an initial residual weight corresponding to the initial residual block; Adjusting a current initial detection weight of the initial detector based on the label loss value and the distillation loss value to obtain an initial detection weight of the initial detector; The initial model weight is obtained based on the initial residual weight and the initial detection weight.

10. The target detection network training method according to claim 9, characterized in that: Generating the reference model weights of the reference target detection network according to the initial model weights includes: Obtaining a current reference model weight of the reference object detection network; An exponential moving average operation is performed based on a preset decay rate, the current reference model weight, and the initial model weight to obtain the reference model weight.

11. The target detection network training method according to claim 1, characterized in that: After calculating the intermediate loss value corresponding to the initial residual block according to the fused feature and the reference intermediate feature, the method further includes: Obtaining a cosine similarity value between the initial image and the enhanced image; If the cosine similarity value is greater than the consistency threshold, calculating an enhanced consistency parameter according to the basic metric value, the cosine similarity value and the consistency threshold; otherwise, using the basic metric value as the enhanced consistency parameter; The intermediate loss value is updated using a product of the enhanced consistency parameter and the intermediate loss value.

12. The target detection network training method according to claim 11, characterized in that: The calculating the enhanced consistency parameter according to the basic metric value, the cosine similarity value and the consistency threshold includes: Calculating a first difference between the cosine similarity value and the consistency threshold, and calculating a second difference between the cosine similarity value and the consistency threshold; Using the first difference as a base and the second difference as a power, a middle value is calculated; A quotient of the intermediate value and the second difference is calculated, and the enhanced consistency parameter is obtained according to the sum of the quotient and the basic metric value.

13. A target detection network training device, characterized in that: The apparatus is performed using an initial target detection network and a reference target detection network having consistent network architectures, wherein the initial target detection network includes an initial feature extraction network and an initial detector, wherein the initial feature extraction network includes a plurality of cascaded initial residual blocks, and the reference target detection network includes a reference feature extraction network and a reference detector, wherein the reference feature extraction network includes a reference residual block corresponding to a position of the initial residual block. Intermediate feature extraction module: used to send the initial image into the reference feature extraction network for feature extraction, obtain the reference intermediate features output by each reference residual block, send the enhanced image corresponding to the initial image into the initial feature extraction network for feature extraction, obtain the initial intermediate features output by each initial residual block, obtain the initial intermediate features and the reference intermediate features corresponding to the position to obtain multiple intermediate feature groups; Target detection module: used to use the last reference intermediate feature as detection position information, input the detection position information into the reference detector to perform target detection to obtain a label result, and input the detection position information into the initial detector to perform target detection to obtain a prediction result, and calculate a label loss value based on the label result and the prediction result; An intermediate loss calculation module is configured to input at least the initial intermediate feature into the corresponding feature fusion network for feature fusion to obtain a fused feature for each intermediate feature group, and calculate the intermediate loss value corresponding to the corresponding initial residual block based on the fused feature and the reference intermediate feature, wherein the input data of the last feature fusion network is the corresponding initial intermediate feature, and the input data of the other feature fusion networks are the initial intermediate feature and the fused feature corresponding to the next feature fusion network; The model weight update module is used to obtain the initial model weight of the initial target detection network based on the label loss value and at least one of the intermediate loss values, generate the reference model weight of the reference target detection network according to the initial model weight, continue to iterate until the end of training, and obtain the trained initial target detection network.

14. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the target detection network training method according to any one of claims 1 to 12 when executing the computer program.

15. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the target detection network training method according to any one of claims 1 to 12 is implemented.

Images