Target detection method, apparatus, device, storage medium, and program product

Abstract

This application discloses a target detection method, apparatus, device, storage medium, and program product. When performing target detection on an image to be detected acquired under target environmental conditions, feature extraction is performed on the image to be detected to obtain proposed features. Based on the similarity between the proposed features and class centers in a class center dataset, target class centers associated with the proposed features are determined from the class center dataset. The class center dataset includes class centers of different categories, which are representative features corresponding to different categories of objects. Therefore, the proposed features are enhanced using the target class centers to obtain enhanced features. Based on the enhanced features, the target image features corresponding to the image to be detected are determined, thereby obtaining richer and more comprehensive features that can represent a certain category of objects, making the target image features richer and more discriminative. Thus, by performing target detection on the image to be detected based on the target image features, the detection results are more accurate, improving the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular, to a target detection method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] With the development of computer technology, more and more scenarios need to be detected, such as unmanned driving, intelligent transportation, security systems, etc.

[0003] In these scenarios, images can be collected by cameras, infrared, thermal, laser, and other optical sensors, and then features are extracted based on the collected images for a classifier to classify the target, so as to find all the targets of interest in the image and determine their categories (such as people, cars, trees, etc.) and positions.

[0004] However, in adverse environmental conditions such as fog, rain, night, etc., the effectiveness of image data obtained by using cameras, infrared, thermal, laser, and other image acquisition devices will be reduced, and it is difficult to obtain enough features for the classifier to classify the target, so that false detection, missed detection, etc. may occur, and the accuracy of target detection is low. SUMMARY

[0005] To solve the above technical problems, the present application provides a target detection method, device, equipment, storage medium and program product, and the detection result is more accurate, and the accuracy of target detection is improved.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In one aspect, the present application provides a target detection method, comprising:

[0008] obtaining a to-be-detected image collected under a target environmental condition;

[0009] performing feature extraction on the to-be-detected image to obtain proposal features;

[0010] determining a target class center associated with the proposal features from a class center data set according to the similarity between the proposal features and the class centers in the class center data set, the class center data set including class centers of different categories, and the class center being a representative feature corresponding to objects of different categories;

[0011] enhancing the proposal features by the target class center to obtain enhanced features;

[0012] determining a target image feature corresponding to the to-be-detected image according to the enhanced features;

[0013] According to the target image feature, target detection is performed on the to-be-detected image, and a detection result is obtained.

[0014] In another aspect, an embodiment of the present application provides a target detection device, which comprises an acquisition unit, an extraction unit, a determination unit, an enhancement unit and a detection unit.

[0015] The acquisition unit is configured to acquire a to-be-detected image collected under a target environment condition.

[0016] The extraction unit is configured to perform feature extraction on the to-be-detected image to obtain a proposal feature.

[0017] The determination unit is configured to determine a target class center associated with the proposal feature from a class center data set according to a similarity between the proposal feature and a class center in the class center data set, wherein the class center data set comprises class centers of different categories, and the class center is a representative feature corresponding to objects of different categories.

[0018] The enhancement unit is configured to enhance the proposal feature by using the target class center to obtain an enhanced feature.

[0019] The determination unit is further configured to determine a target image feature corresponding to the to-be-detected image according to the enhanced feature.

[0020] The detection unit is configured to perform target detection on the to-be-detected image according to the target image feature, and obtain a detection result.

[0021] In another aspect, an embodiment of the present application provides a device for target detection, which comprises a processor and a memory.

[0022] The memory is configured to store program code and transmit the program code to the processor.

[0023] The processor is configured to execute the foregoing target detection method according to instructions in the program code.

[0024] In another aspect, an embodiment of the present application provides a computer readable storage medium for storing program code, wherein the program code is used to execute the foregoing target detection method.

[0025] In another aspect, an embodiment of the present application provides a computer program product, which comprises a computer program and is used to implement the foregoing target detection method when the computer program is executed by a processor.

[0026] It can be seen from the technical solution that the application pre-collects class centers of different categories, the class center is a representative feature corresponding to an object of a different category and can reflect a prototype of a category. When target detection is performed on a to-be-detected image collected under a target environment condition, a proposal feature can be extracted from the to-be-detected image, and then a target class center associated with the proposal feature can be determined from a class center data set according to the similarity between the proposal feature and the class center in the class center data set. The class center data set includes class centers of different categories, and the class center is a representative feature corresponding to an object of a different category. Therefore, the target class center can be used to enhance the proposal feature to obtain an enhanced feature, and a target image feature corresponding to the to-be-detected image can be determined according to the enhanced feature, so that a feature that is more comprehensive and can represent an object of a certain category is obtained. Especially when the target environment condition destroys the to-be-detected image, resulting in insufficient and lack of distinguishability of the extracted proposal feature, the target class center is used to enhance the proposal feature to assist the target detection process, so that the target image feature obtained is more comprehensive and has distinguishability. In this way, target detection is performed on the to-be-detected image according to the target image feature, and the detection result obtained is more accurate, thereby improving the accuracy of target detection. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0028] Figure 1 A system architecture schematic diagram of a target detection method provided by an embodiment of the application;

[0029] Figure 2 A flowchart of a target detection method provided by an embodiment of the application;

[0030] Figure 3 A flowchart of a target detection method provided by an embodiment of the application;

[0031] Figure 4 A flowchart of a training method of a target detection model provided by an embodiment of the application;

[0032] Figure 5 A process example diagram of updating a class center of a category to which an object belongs in a class center module (memory feature library) provided by an embodiment of the application;

[0033] Figure 6 A structural diagram of a target detection device provided by an embodiment of the application;

[0034] Figure 7 A structural diagram of a terminal device provided for an embodiment of the present application is shown in FIG. 1.

[0035] Figure 8 A structural diagram of a server provided for an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0036] Embodiments of the present application are described below in conjunction with the accompanying drawings.

[0037] Due to the extreme environment such as haze, rainy day, night, etc., the features of the extracted object (e.g., proposal features) can lack discriminability, such as a car in thick fog being easily ignored or identified as a bus or other vehicle.

[0038] In order to cope with the poor visibility condition such as haze, rainy day, night, etc., to solve the technical problem of low accuracy of target detection under the harsh environmental conditions such as haze, rainy day, night, etc., an embodiment of the present application provides a target detection method.

[0039] Since objects of the same category (e.g., people, objects, etc.) must be similar in some aspects, different aspects here represent different feature spaces, such as size, shape, etc. By integrating the features of multiple feature spaces, the object can be correctly classified, for example, by distinguishing different categories such as cars, trucks, etc. through the shape and size of the object of the vehicle. Therefore, in the embodiment of the present application, the class centers of different categories can be collected in advance. The class center is a representative feature corresponding to an object of different categories, which can reflect the prototype of a category, and then the class center is used to enhance the recognition process. For example, the proposal features are obtained by performing feature extraction on the to-be-detected image collected under the target environmental condition, and then the proposal features are enhanced by the target class center, so as to improve the discriminative representation of the to-be-detected image damaged by the target environmental condition, so that the final target image feature is more rich and has discriminability, thereby improving the accuracy of target detection.

[0040] It should be noted that the method provided by the embodiment of the present application can be applied to various scenes, especially scenes where image collection is affected by environmental conditions, such as unmanned driving, intelligent transportation, security systems, etc. The embodiment of the present application does not limit this.

[0041] Reference is made to Figure 1 , Figure 1 A system architecture diagram of the target detection method provided by the embodiment of the present application is shown in FIG. 3. The system architecture includes an image collection device 101 and an image processing device 102. The image collection device 101 can be a camera, infrared, thermal, laser, etc. optical sensor, Figure 1The image acquisition device 101 is mainly taken as a camera for example. The image processing device 102 can be a terminal device or a server, Figure 1 The image processing device 102 is mainly taken as a terminal device for example. The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a vehicle-mounted terminal, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0042] In the security system scenario, in order to improve security, image acquisition devices 101 are usually installed at different locations. For example, an image acquisition device 101 is installed at an intersection. The image acquisition device 101 can capture the scene near the intersection (see FIG. 1B) to obtain a to-be-detected image. The scene near the intersection includes people, cars, etc. Figure 1 The to-be-detected image is captured by the image acquisition device under the target environment condition, which includes various categories of objects, such as people, cars, etc. It can be understood that the categories mainly include people, cars, buses, bicycles, motorcycles, traffic lights, etc.

[0043] Since the image acquisition device 101 captures the outdoor scene, the to-be-detected image captured may be affected by the environment condition. If the to-be-detected image is captured under the target environment condition (e.g., rainy day), the image acquisition device 101 can send the to-be-detected image to the image processing device 102 for target detection.

[0044] In the process of target detection, the image processing device 102 can first perform feature extraction on the to-be-detected image to obtain a proposal feature. At this time, since the to-be-detected image may be damaged by the target environment condition, the proposal feature may lack distinguishability. Therefore, the target class center associated with the proposal feature can be determined from the class center data set according to the similarity between the proposal feature and the class center in the class center data set. The class center data set includes class centers of different categories. The class center is a representative feature corresponding to an object of a different category. Therefore, the target class center can be used to enhance the proposal feature to obtain an enhanced feature. The target image feature corresponding to the to-be-detected image is determined according to the enhanced feature, so that a feature that is more comprehensive and can represent an object of a certain category is obtained. In this way, the to-be-detected image is detected according to the target image feature, and the detection result is more accurate, thereby improving the accuracy of target detection.

[0045] It should be noted that, in some cases, the method provided in the embodiments of this application can also be executed jointly by a terminal device and a server, that is, the image processing device 102 includes a terminal device and a server.

[0046] The target detection method provided in the embodiments of this application will now be described in conjunction with the accompanying drawings. See also... Figure 2 The method includes:

[0047] S201. Acquire the image to be detected under the target environmental conditions.

[0048] In real-world scenarios, image acquisition devices may capture images under various environmental conditions. However, these environmental conditions can be quite harsh, potentially damaging the acquired images. The method provided in this application embodiment allows for target detection on images acquired under these environmental conditions, yielding more accurate detection results. This application embodiment uses environmental conditions as the target environmental conditions as an example; images acquired under these target environmental conditions for target detection can be referred to as the images to be detected.

[0049] S202, Perform feature extraction on the image to be detected to obtain proposed features.

[0050] It should be noted that, in the embodiments of this application, after obtaining the image to be detected, the image to be detected can be input into the target detection model so that the target detection model can be used to perform target detection on the image to be detected and obtain the detection result.

[0051] The object detection model comprises a feature extraction module, a class center module, a feature fusion module, and a recognition module. The class center module consists of class centers from the class center dataset. Detecting objects in an image using this model mainly involves two stages. The first stage primarily involves feature extraction, such as using the feature extraction module to extract proposed features from the image. The second stage involves subsequent recognition and classification based on the extracted proposed features. This second stage primarily utilizes the class center module, feature fusion module, and recognition module.

[0052] First, let's introduce the first stage. See [link / reference] Figure 3As shown, the feature extraction module used in the first stage mainly includes a Region Proposal Network (RPN) and a Region of Interest (ROI) alignment module, such as ROI Align or ROI pooling. First, the RPN in the first stage obtains the possible locations of objects in the image to be detected, resulting in region proposal boxes. Then, these region proposal boxes are subjected to non-maximum suppression to remove some region proposal boxes that repeatedly appear for the same object. These region proposal boxes are then processed by ROI pooling or ROI Align to obtain corresponding proposal features (e.g., ...). Figure 3 As shown in Figure 301), the second stage of recognition is then performed to conduct final classification and regression operations to obtain the final category and location.

[0053] Based on this, the method provided in the embodiments of this application may involve artificial intelligence (AI). Artificial intelligence is the simulation, extension and expansion of human intelligence using digital computers or machines controlled by digital computers. Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning and decision-making.

[0054] The method provided in this application also relates to computer vision (CV) technology in artificial intelligence. Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. Computer vision technology typically includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, and other technologies. It also includes common biometric recognition technologies such as face recognition and fingerprint recognition. For example, proposed features are obtained by extracting features from the image to be detected through image semantic understanding.

[0055] S203. Based on the similarity between the proposed feature and the class centers in the class center dataset, determine the target class center associated with the proposed feature from the class center dataset.

[0056] In this embodiment, a class center dataset can be pre-constructed, which includes class centers for different categories. Each class center represents a representative feature of an object of a different category. This class center dataset can constitute a class center module, which can also be called a memory feature bank.

[0057] The class centers in the class center dataset can be collected during the training of the object detection model. During the training of the object detection model, representative features of each category of objects are continuously collected as the class centers of that category. The class centers not only participate in the training process, but will also be saved to help with object detection.

[0058] Since the image to be detected may be corrupted by the target environment, the proposed features may lack discriminative power. Therefore, target class centers associated with the proposed features can be selected from the class center dataset based on the similarity between the proposed features and the class centers in the class center dataset. See [link to relevant documentation]. Figure 3 As shown in the dashed box 302. For example, the similarity between the proposed feature and each class center in the class center dataset is calculated. The higher the similarity, the more likely the object corresponding to that class center and the object corresponding to the proposed feature are of the same category. Therefore, class centers with similarity higher than a certain threshold can be used as target class centers. Since the class center dataset includes class centers of different categories, and class centers are representative features corresponding to objects of different categories, the determined target class centers may be representative features of the objects corresponding to the proposed features. Therefore, the target class centers can be used to enhance the proposed features to obtain enhanced features, thereby obtaining richer and more comprehensive features that can represent objects of a certain category.

[0059] It should be noted that when using the above-mentioned target detection model for target detection, in this embodiment, the target class center associated with the proposed feature can be determined through the class center module in the target detection model.

[0060] In one possible implementation, for the same object, different environmental conditions may cause its corresponding representative features to differ. Therefore, in this embodiment, different class center datasets corresponding to different environmental conditions can be constructed. In this case, S203 can be implemented by obtaining the class center dataset corresponding to the target environmental conditions and determining the target class centers associated with the proposed features from the class center dataset corresponding to the target environmental conditions.

[0061] For example, class center datasets corresponding to haze, rainy days, and nighttime can be constructed respectively. If the image to be detected was collected on a rainy day, then when determining the target class center, it is necessary to calculate the similarity between the proposed features and the class centers in the class center dataset corresponding to haze, and then determine the target class center from the class center dataset corresponding to haze.

[0062] Compared with related technologies that utilize invariant feature domains under various environmental conditions to enhance the feature representation of objects, this method takes into account the inherent attributes under different environmental conditions. The target class center under the same environmental conditions can provide better guidance information, further improving the accuracy of subsequent target detection.

[0063] S204. Enhance the proposed features by using the target class center to obtain enhanced features.

[0064] Generally, class centers of the same category (target class centers) provide better guidance information. The feature fusion module in this embodiment can be an attention-based feature fusion module, which uses the commonly used dot product attention formula to enhance the proposed features. In one possible implementation, a similarity measurement matrix can be calculated based on the target class centers and the proposed features. The elements in the similarity measurement matrix are used to identify the weight value of each target class center for the proposed features. Then, the target class centers and proposed features are weighted and summed according to the similarity measurement matrix to obtain the enhanced features corresponding to the proposed features. The formula for calculating the similarity measurement matrix is ​​as follows:

[0065] W=softmax(P*C T / d 1 / 2 )

[0066] Where P is the proposed feature, with a dimension of n*d, where n is the number of features in P (the number of objects in the image to be detected), and d is the feature dimension, typically 1024; C is the target class center, C0... T Let C be the transpose; W is the similarity matrix with dimensions [n*m], representing the weight of each target class center with respect to the proposed feature P, where n represents the number of proposed features and m represents the number of class centers in the class center module; softmax is a logistic regression function. Next, weighted summation is used to construct enhanced features for each object's proposed features. To avoid losing the object's original features, enhancements can be made based on the original features (proposed features), as shown in the following formula:

[0067] E = W*C + P

[0068] Where E is the enhanced feature, W is the similarity measurement matrix, C is the target class center, and P is the proposed feature obtained from feature extraction.

[0069] It should be noted that, when using the above-described object detection model for object detection, in this embodiment, the enhanced features can be obtained by fusing the target class center and the proposed features through the feature fusion module in the object detection module. See [link to relevant documentation]. Figure 3The dashed box 303 is shown in the diagram. The dashed box 303 encompasses the entire process of feature fusion performed by the feature fusion module. This process includes three fully connected layers (FC), matrix multiplication, and element-wise addition. From... Figure 3 It can be seen that feature enhancement based on the attention mechanism is achieved through three fully connected layers, two matrix multiplication operations, and one element-wise addition operation, resulting in enhanced features (e.g., ...). Figure 3 (As shown in 304).

[0070] S205. Determine the target image features corresponding to the image to be detected based on the enhanced features.

[0071] It should be noted that when using the above-mentioned target detection model for target detection, in this embodiment, the target image features corresponding to the image to be detected can be determined by the feature fusion module based on the fused features.

[0072] In this application embodiment, the method of determining the target image features corresponding to the image to be detected based on the enhanced features can include multiple methods. One method is to directly use the enhanced features as the target image features, and another method is to concatenate the enhanced features with the proposed features to obtain the target image features. See [link to relevant documentation]. Figure 3 As shown in the dashed box 305, the enhanced features and proposed features are concatenated using a concatenation process to obtain the target image features (e.g., Figure 3 As shown in Figure 306, this avoids the loss of the original information in the image to be detected. If the dimension of the proposed feature is n*d, then the dimension of the obtained target image feature is [n*2d].

[0073] S206. Target detection is performed on the image to be detected based on the target image features to obtain the detection result.

[0074] When using the above-described object detection model for object detection, in this embodiment, the recognition module can perform object detection on the image to be detected based on the features of the target image to obtain the detection result. The detection result may include the category of the object included in the image to be detected and the location of the object in the image to be detected.

[0075] See Figure 3 As shown, in Figure 3 In this context, CLS stands for Category Recognition, which identifies the categories of objects in the image to be detected. REG stands for Regression Location, which determines the location of objects of each category in the image to be detected. See also... Figure 4 The location marked by the rectangle in the middle.

[0076] As can be seen from the above technical solution, this application pre-collects class centers of different categories. Class centers are representative features corresponding to different categories of objects and can reflect the prototype of a category. When performing target detection on an image to be detected acquired under target environmental conditions, feature extraction can be performed on the image to be detected to obtain proposed features. Then, based on the similarity between the proposed features and the class centers in the class center dataset, the target class center associated with the proposed features is determined from the class center dataset. The class center dataset includes class centers of different categories. Class centers are representative features corresponding to different categories of objects. Therefore, the proposed features can be enhanced using the target class centers to obtain enhanced features. Based on the enhanced features, the target image features corresponding to the image to be detected are determined, thereby obtaining richer and more comprehensive features that can represent a certain category of objects. Especially when the target environmental conditions damage the image to be detected, resulting in insufficient extracted proposed features and a lack of discriminability, enhancing the proposed features with target class centers assists the target detection process, making the final target image features richer and more discriminative. In this way, target detection is performed on the image to be detected based on the target image features, resulting in more accurate detection results and improving the accuracy of target detection.

[0077] In one possible implementation, to improve the accuracy of target detection under harsh environmental conditions, an image restoration model can be pre-trained. After the image is acquired, the image restoration model is used to restore the acquired image, thereby improving the quality of the image input to the target detection model and thus increasing the accuracy of target detection. Compared with the approach of improving target detection accuracy based on an image restoration model, the target detection method provided in this application does not require training an additional model, reduces the amount of data processing, and is more suitable for real-world scenarios.

[0078] Tables 1-4 show the accuracy of target detection for different categories of objects using various methods under different environmental conditions. For example, Table 1 introduces the accuracy of identifying categories such as cars, buses, people, bicycles, and motorcycles under hazy environmental conditions using methods such as Domain Adaptive Improvement (DAFaster), Strong-Weak Distribution Alignment for Adaptive Object Detection (SWDA), PBDA, Mask Region-CNN Convolutional Neural Network (MaskRcnn), and MaskRcnn + this scheme (i.e., using MaskRcnn as a baseline and inserting the aforementioned class center module and feature fusion module into MaskRcnn). The table also shows the mean average precision (mAP) for each category under each method. As can be seen from the values ​​in Table 1, the values ​​corresponding to the method combined with this scheme are larger, indicating that the target detection accuracy is higher. Similarly, Tables 2, 3, and 4 show the accuracy rates of different methods for identifying different categories under rainy, foggy, and nighttime environmental conditions, as well as the average accuracy rate of each category under each method. The methods used in Tables 2-4 are different from those in Table 1, and the categories they target are also different, but the values ​​corresponding to the methods in this scheme are all relatively large, that is, the accuracy rate of target detection is relatively high.

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] Table 3

[0084]

[0085] Table 4

[0086]

[0087] Table 5 shows the accuracy of target detection for targets of different sizes using various methods under different environmental conditions, as well as the recognition rate for 100 targets (the proportion of 100 targets that can be recognized out of 100 targets).

[0088] Table 5

[0089]

[0090] As can be seen from Table 5, the numerical values ​​corresponding to the method of this scheme are relatively large, which means that the accuracy of target detection and the recognition rate of this scheme are relatively high under harsh environmental conditions.

[0091] Experiments were conducted using different methods, and the implementation results shown in Tables 1-5 above were obtained. It can be seen that the target detection method provided in this application embodiment can achieve state-of-the-art performance for unrestricted object detection tasks, and the method has better robustness.

[0092] Next, the training method for the object detection model will be introduced. See [link to relevant documentation]. Figure 2 The method further includes:

[0093] S401. Obtain training sample images under sample environment conditions, wherein the training sample images have category labels for the objects they include.

[0094] S402. Input the training sample images into the initial detection model.

[0095] The initial detection model includes an initial feature extraction module, an initial class center module, an initial feature fusion module, and an initial recognition module. The training sample images have the category labels of the objects they contain.

[0096] S403. The initial feature extraction module extracts features from the training sample images to obtain sample proposal features.

[0097] S404. Using the initial class center module, based on the similarity between the sample proposal features and the class centers in the class center dataset, determine the target sample class center associated with the sample proposal features from the initial class center module.

[0098] S405. Through the initial feature fusion module, the sample proposal features are enhanced using the target sample class center to obtain sample enhancement features.

[0099] S406. The initial feature fusion module determines the target sample image features corresponding to the training sample image based on the sample enhancement features.

[0100] S407. The initial recognition module performs target detection on the image to be detected based on the target sample image features to obtain the sample detection result.

[0101] S408. The initial detection model is trained based on the category label and the sample detection results to obtain the target detection model.

[0102] It should be noted that during the training of the object detection model, S401-S407 and Figure 5The corresponding target detection process is similar and will not be described in detail here.

[0103] Based on this, the method provided in this application also relates to the field of machine learning. Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning. In this application embodiment, a target detection model is trained using machine learning methods.

[0104] During the training of the object detection model, a class center module can be constructed. This can be done by using a feature extraction module to extract features from the training sample images during model training, obtaining sample proposal features for objects identified by category labels. The category labels are used to identify the categories of objects included in the training sample images. The class centers of the objects in the class center module are then updated based on these sample proposal features.

[0105] In the training process of the target detection model, this application embodiment introduces a class center module to improve the discriminative representation of damaged objects through representative features in the class center module, which is beneficial for learning a robust target detection model.

[0106] It should be noted that updating the class centers of the object's category in the class center module based on the object's sample proposal features can be done by using the object's sample proposal features as the class centers of the object's category in the class center module. However, since the class centers of the category in the class center module require a certain amount of storage space, the more class centers it contains, the more storage space it occupies, thus wasting storage resources to some extent and potentially affecting the performance of object detection. Therefore, in one possible implementation, updates can be based on the number of class centers of the object's category in the class center module. Typically, if the number of class centers of the object's category is zero (i.e., the set of class centers Ck corresponding to this category is empty), then the object's sample proposal features are used as the class centers of the object's category; if the number of class centers of the object's category is not zero (i.e., the set of class centers Ck corresponding to this category is not empty), then the similarity between the object's sample proposal features and the class centers of the object's category is calculated, and the class centers of the object's category in the class center module are updated based on the similarity. The similarity between the object's sample proposal features and the class centers of the object's category can be cosine similarity.

[0107] In some possible implementations, if the similarity is less than a similarity threshold (e.g., a similarity threshold set to 0.6), the sample proposal features of the object are directly added to the class center of the object's category. The similarity threshold can be set according to actual needs; this embodiment does not limit its specific value.

[0108] To avoid an excessive number of class centers in the class center module due to the addition of proposed features, one possible implementation is to update the class centers of the object's category in the class center module based on similarity. If the similarity is less than a similarity threshold, the relationship between the number of class centers of the object's category and the threshold is determined, and then the class centers of the object's category in the class center module are updated based on this relationship.

[0109] Typically, if the number of class centers belonging to the same category as an object in the class center module reaches a threshold (which can be set to 50), it indicates that the class center module is full. In this case, the class center with the highest similarity can be replaced by a proposed feature. The threshold can be set according to actual needs; this embodiment does not limit its specific value.

[0110] In this way, while obtaining representative features of the object, the number of class centers can be reduced as much as possible, thereby reducing the storage space occupied and ensuring the target detection performance.

[0111] In this way, as the object detection model is trained, a class center module containing representative features can be obtained, and this acquisition method allows the entire class center module to be trained end-to-end along with the object detection model.

[0112] See Figure 5 As shown, Figure 5 This diagram illustrates the process of updating the class centers of objects within the class center module (memory feature library). For each training sample image in a minibatch, all bounding boxes (also known as ground-truth boxes) are obtained using the ground-truth class labels. ROIAlign is then used to obtain proposed features for the objects corresponding to those boxes. Each proposed feature has its own true class label, and the obtained proposed features are used to update the class center corresponding to its own class label. Figure 5 In the diagram, the category labels include category label 1, category label 2, category label 3, ... The number of class centers for label 1 has reached a threshold. Since the proposed feature corresponding to the new label 1 has low similarity to all the class centers in the class corresponding to label 1, the most similar class center (e.g., ...) is replaced by the proposed feature corresponding to the new label 1. Figure 5 (As shown in 501). The number of class centers in label2 did not reach the threshold, and the similarity between the newly proposed feature of label2 and all the class centers in label2 was below the similarity threshold. Therefore, this proposed feature was placed into the class center of label2 (e.g., Figure 6 (As shown in 502). Although the number of class centers of label3 did not reach the threshold, the proposed features of the newly arrived label3 were very similar to all the class centers of label3, so the proposed features of label3 were directly discarded.

[0113] It should be noted that when updating the class centers of objects within the class center module, a target loss function can be constructed to supervise the enhancement features in order to obtain enhanced features E with better representation and discriminability. For example, a target loss function can be constructed based on the difference between the class label and the obtained sample enhancement features, and the class centers in the class center module can be updated according to the target loss function to construct the class center module.

[0114] The target loss function can be of different types, such as L2 normloss, L2 loss function, L1 loss function, etc. Taking L2 normloss as an example, its formula can be shown below:

[0115] Ll2 = L2norm(L*Lt – E*Et)

[0116] Where L is the class label, L2norm is L2 normalization, E is the sample augmentation feature, and Lt and Et are coefficients.

[0117] In this embodiment of the application, when updating the class center of the category to which an object belongs in the class center module, a target loss function for structural embedding is designed for it, so as to dynamically store representative features and accurately propagate features to improve feature differentiation.

[0118] Based on the target detection method provided in the foregoing embodiments, this application also provides a target detection device 600. See also... Figure 7 The device 600 includes an acquisition unit 601, an extraction unit 602, a determination unit 603, an enhancement unit 604, and a detection unit 605.

[0119] The acquisition unit 601 is used to acquire the image to be detected under the target environmental conditions;

[0120] The extraction unit 602 is used to extract features from the image to be detected to obtain proposed features;

[0121] The determining unit 603 is used to determine a target class center associated with the proposed feature from the class center dataset based on the similarity between the proposed feature and the class centers in the class center dataset. The class center dataset includes class centers of different categories, and the class centers are representative features corresponding to objects of different categories.

[0122] The enhancement unit 604 is used to enhance the proposed features through the target class center to obtain enhanced features;

[0123] The determining unit 603 is further configured to determine the target image features corresponding to the image to be detected based on the enhanced features;

[0124] The detection unit 605 is used to perform target detection on the image to be detected based on the target image features, and obtain the detection result.

[0125] In one possible implementation, the apparatus further includes an input unit for inputting the image to be detected into a target detection model. The target detection model includes a feature extraction module, a class center module, a feature fusion module, and a recognition module. The class center module is composed of class centers from the class center dataset.

[0126] The extraction unit 602 is used to extract the proposed features from the image to be detected through the feature extraction module.

[0127] The determining unit 603 is used to determine the target class center associated with the proposed feature through the class center module;

[0128] The enhancement unit 604 is used to fuse the target class center and the proposed feature through the feature fusion module to obtain the enhanced feature;

[0129] The determining unit 603 is further configured to determine the target image features corresponding to the image to be detected based on the fused features through the feature fusion module;

[0130] The detection unit 605 is used to perform target detection on the image to be detected by the recognition module according to the target image features, and obtain the detection result.

[0131] In one possible implementation, the apparatus further includes a construction unit for constructing the class center module, wherein constructing the class center module includes:

[0132] During the training of the target detection model, the feature extraction module extracts features from the training sample images to obtain sample proposal features of objects identified by category labels. The category labels are used to identify the categories of objects included in the training sample images.

[0133] The class center of the category to which the object belongs in the class center module is updated based on the sample proposal features of the object.

[0134] In one possible implementation, the building unit is further used for:

[0135] If the number of class centers of the category to which the object belongs is zero, then the sample proposal feature of the object is used as the class center of the category to which the object belongs;

[0136] If the number of class centers of the category to which the object belongs is not zero, then calculate the similarity between the sample proposal features of the object and the class centers of the category to which the object belongs;

[0137] The class center of the category to which the object belongs in the class center module is updated based on the similarity.

[0138] In one possible implementation, the building unit is further used for:

[0139] If the similarity is less than the similarity threshold, the sample proposal features of the object are added to the class center of the category to which the object belongs.

[0140] In one possible implementation, the building unit is further used for:

[0141] If the similarity is less than the similarity threshold, determine the relationship between the number of class centers of the category to which the object belongs and the number threshold;

[0142] Update the class center of the category to which the object belongs in the class center module according to the size relationship.

[0143] In one possible implementation, the building unit is further used for:

[0144] Construct a target loss function based on the difference between the category labels and the obtained sample enhancement features;

[0145] The class centers in the class center module are updated according to the target loss function to construct the class center module.

[0146] In one possible implementation, different class center datasets correspond to different environmental conditions, and the determining unit 603 is used for:

[0147] Obtain the class center dataset corresponding to the target environmental conditions;

[0148] The target class center associated with the proposed feature is determined from the class center dataset corresponding to the target environmental conditions.

[0149] In one possible implementation, the determining unit 603 is used to:

[0150] The enhanced features are concatenated with the proposed features to obtain the target image features.

[0151] In one possible implementation, the enhancement unit 604 is used for:

[0152] A similarity measurement matrix is ​​calculated based on the target class center and the proposed feature, wherein the elements in the similarity measurement matrix are used to identify the weight value of each target class center for the proposed feature;

[0153] The target class center and the proposed feature are weighted and summed according to the similarity measurement matrix to obtain the enhanced feature corresponding to the proposed feature.

[0154] In one possible implementation, the apparatus further includes a training unit, the training unit being configured to:

[0155] Acquire training sample images under sample environment conditions, wherein the training sample images have category labels for the objects they include;

[0156] The training sample images are input into an initial detection model, which includes an initial feature extraction module, an initial class center module, an initial feature fusion module, and an initial recognition module. The training sample images have category labels for the objects they contain.

[0157] The initial feature extraction module extracts features from the training sample images to obtain sample proposal features;

[0158] The initial class center module determines the target sample class center associated with the sample proposal features based on the similarity between the sample proposal features and the class centers in the class center dataset.

[0159] The initial feature fusion module enhances the proposed features of the samples using the target sample class center to obtain enhanced sample features.

[0160] The initial feature fusion module determines the target sample image features corresponding to the training sample image based on the sample enhancement features.

[0161] The initial recognition module performs target detection on the image to be detected based on the features of the target sample image to obtain the sample detection result.

[0162] The initial detection model is trained based on the category label and the sample detection results to obtain the target detection model.

[0163] As can be seen from the above technical solution, this application pre-collects class centers of different categories. Class centers are representative features corresponding to different categories of objects and can reflect the prototype of a category. When performing target detection on an image to be detected acquired under target environmental conditions, feature extraction can be performed on the image to be detected to obtain proposed features. Then, based on the similarity between the proposed features and the class centers in the class center dataset, the target class center associated with the proposed features is determined from the class center dataset. The class center dataset includes class centers of different categories. Class centers are representative features corresponding to different categories of objects. Therefore, the proposed features can be enhanced using the target class centers to obtain enhanced features. Based on the enhanced features, the target image features corresponding to the image to be detected are determined, thereby obtaining richer and more comprehensive features that can represent a certain category of objects. Especially when the target environmental conditions damage the image to be detected, resulting in insufficient extracted proposed features and a lack of discriminability, enhancing the proposed features with target class centers assists the target detection process, making the final target image features richer and more discriminative. In this way, target detection is performed on the image to be detected based on the target image features, resulting in more accurate detection results and improving the accuracy of target detection.

[0164] Based on the above embodiments, this application also provides a device for target detection. This device can be the aforementioned image processing device, and therefore the image processing device can be a terminal device, taking a smartphone as an example:

[0165] Figure 7 This is a block diagram illustrating a portion of the structure of a smartphone related to the terminal provided in the embodiments of this application. (Reference) Figure 7 The smartphone includes components such as: a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless fidelity (WiFi) module 770, a processor 780, and a power supply 790. The input unit 730 may include a touch panel 731 and other input devices 732, the display unit 740 may include a display panel 741, and the audio circuit 760 may include a speaker 761 and a microphone 762. Those skilled in the art will understand that... Figure 8 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0166] The memory 720 can be used to store software programs and modules. The processor 780 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0167] The processor 780 is the control center of the smartphone, connecting various parts of the smartphone via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 720, and by accessing data stored in the memory 720. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 780.

[0168] In this embodiment, the processor 780 in the smartphone can perform the following steps:

[0169] Acquire the image to be detected under the target environmental conditions;

[0170] Proposed features are obtained by extracting features from the image to be detected;

[0171] Based on the similarity between the proposed feature and the class centers in the class center dataset, target class centers associated with the proposed feature are determined from the class center dataset. The class center dataset includes class centers of different categories, and the class centers are representative features corresponding to objects of different categories.

[0172] Enhanced features are obtained by enhancing the proposed features using the target class center;

[0173] The target image features corresponding to the image to be detected are determined based on the enhanced features;

[0174] The target image is used to perform target detection on the image to be detected, and the detection result is obtained.

[0175] The image processing device provided in this application embodiment can also be a server; please refer to [link to relevant documentation]. Figure 8As shown, ​ This is a structural diagram of a server 800 provided in an embodiment of this application. The server 800 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 822 (e.g., one or more processors) and a memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 842 or data 844. The memory 832 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the central processing unit 822 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the server 800.

[0176] Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input / output interfaces 858, and / or one or more operating systems 841, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0177] In this embodiment, the central processing unit 822 in the server 800 can perform the following steps:

[0178] Acquire the image to be detected under the target environmental conditions;

[0179] Proposed features are obtained by extracting features from the image to be detected;

[0180] Based on the similarity between the proposed feature and the class centers in the class center dataset, target class centers associated with the proposed feature are determined from the class center dataset. The class center dataset includes class centers of different categories, and the class centers are representative features corresponding to objects of different categories.

[0181] Enhanced features are obtained by enhancing the proposed features using the target class center;

[0182] The target image features corresponding to the image to be detected are determined based on the enhanced features;

[0183] The target image is used to perform target detection on the image to be detected, and the detection result is obtained.

[0184] According to one aspect of this application, a computer-readable storage medium is provided for storing program code for performing the target detection methods described in the foregoing embodiments.

[0185] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0186] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0188] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0190] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A target detection method, characterized in that, The method includes: Acquire the image to be detected under the target environmental conditions; Feature extraction is performed on the image to be detected to obtain proposed features; this includes obtaining the object locations in the image to be detected through a region proposal network to obtain region proposal boxes, and then obtaining the corresponding proposed features from the region proposal boxes through a region of interest alignment module. Based on the similarity between the proposed feature and the class centers in the class center dataset, target class centers associated with the proposed feature are determined from the class center dataset. The class center dataset includes class centers of different categories, and the class centers are representative features corresponding to objects of different categories. A similarity measurement matrix is ​​calculated based on the target class center and the proposed feature, wherein the elements in the similarity measurement matrix are used to identify the weight value of each target class center for the proposed feature; The target class center and the proposed features are weighted and summed according to the similarity measurement matrix to obtain the enhanced features corresponding to the proposed features; The target image features corresponding to the image to be detected are determined based on the enhanced features; The target image is used to perform target detection on the image to be detected, and the detection result is obtained.

2. The method according to claim 1, characterized in that, The method further includes: The image to be detected is input into the target detection model, which includes a feature extraction module, a class center module, a feature fusion module, and a recognition module. The class center module is composed of class centers from the class center dataset. The step of extracting proposed features from the image to be detected includes: The proposed features are obtained by extracting features from the image to be detected using the feature extraction module. The step of determining the target class center associated with the proposed feature from the class center dataset based on the similarity between the proposed feature and the class centers in the class center dataset includes: The target class center associated with the proposed feature is determined by the class center module; The enhanced features are obtained by enhancing the proposed features through the target class center, including: The enhanced feature is obtained by fusing the target class center and the proposed feature through the feature fusion module. Determining the target image features corresponding to the image to be detected based on the enhanced features includes: The feature fusion module determines the target image features corresponding to the image to be detected based on the fused features. The step of performing target detection on the image to be detected based on the target image features to obtain detection results includes: The recognition module performs target detection on the image to be detected based on the target image features to obtain the detection result.

3. The method according to claim 2, characterized in that, The method further includes constructing the class center module, wherein constructing the class center module includes: During the training of the target detection model, the feature extraction module extracts features from the training sample images to obtain sample proposal features of objects identified by category labels. The category labels are used to identify the categories of objects included in the training sample images. The class center of the category to which the object belongs in the class center module is updated based on the sample proposal features of the object.

4. The method according to claim 3, characterized in that, The step of updating the class center of the category to which the object belongs in the class center module according to the sample proposal features of the object includes: If the number of class centers of the category to which the object belongs is zero, then the sample proposal feature of the object is used as the class center of the category to which the object belongs; If the number of class centers of the category to which the object belongs is not zero, then calculate the similarity between the sample proposal features of the object and the class centers of the category to which the object belongs; The class center of the category to which the object belongs in the class center module is updated based on the similarity.

5. The method according to claim 4, characterized in that, Updating the class center of the category to which the object belongs in the class center module according to the similarity includes: If the similarity is less than the similarity threshold, the sample proposal features of the object are added to the class center of the category to which the object belongs.

6. The method according to claim 4, characterized in that, Updating the class center of the category to which the object belongs in the class center module according to the similarity includes: If the similarity is less than the similarity threshold, determine the relationship between the number of class centers of the category to which the object belongs and the number threshold; Update the class center of the category to which the object belongs in the class center module according to the size relationship.

7. The method according to claim 3, characterized in that, The construction of the class center module includes: Construct a target loss function based on the difference between the category labels and the obtained sample enhancement features; The class centers in the class center module are updated according to the target loss function to construct the class center module.

8. The method according to any one of claims 1-7, characterized in that, Different class center datasets correspond to different environmental conditions. The step of determining the target class center associated with the proposed feature from the class center dataset based on the similarity between the proposed feature and the class centers in the class center dataset includes: Obtain the class center dataset corresponding to the target environmental conditions; The target class center associated with the proposed feature is determined from the class center dataset corresponding to the target environmental conditions.

9. The method according to any one of claims 1-7, characterized in that, Determining the target image features corresponding to the image to be detected based on the enhanced features includes: The enhanced features are concatenated with the proposed features to obtain the target image features.

10. The method according to any one of claims 2-7, characterized in that, The method further includes: Acquire training sample images under sample environment conditions, wherein the training sample images have category labels for the objects they include; The training sample images are input into an initial detection model, which includes an initial feature extraction module, an initial class center module, an initial feature fusion module, and an initial recognition module. The training sample images have category labels for the objects they contain. The initial feature extraction module extracts features from the training sample images to obtain sample proposal features; The initial class center module determines the target sample class center associated with the sample proposal features based on the similarity between the sample proposal features and the class centers in the class center dataset. The initial feature fusion module enhances the proposed features of the samples using the target sample class center to obtain enhanced sample features. The initial feature fusion module determines the target sample image features corresponding to the training sample image based on the sample enhancement features. The initial recognition module performs target detection on the image to be detected based on the features of the target sample image to obtain the sample detection result. The initial detection model is trained based on the category label and the sample detection results to obtain the target detection model.

11. A target detection device, characterized in that, The device includes an acquisition unit, an extraction unit, a determination unit, an enhancement unit, and a detection unit: The acquisition unit is used to acquire the image to be detected under the target environmental conditions; The extraction unit is used to extract features from the image to be detected to obtain proposed features; This includes obtaining the location of objects in the image to be detected through a region proposal network, obtaining region proposal boxes, and then using the region proposal boxes to obtain corresponding proposal features through a region of interest alignment module; The determining unit is configured to determine, based on the similarity between the proposed feature and the class centers in the class center dataset, a target class center associated with the proposed feature, wherein the class center dataset includes class centers of different categories, and the class centers are representative features corresponding to objects of different categories. The enhancement unit is configured to calculate a similarity measurement matrix based on the target class center and the proposed feature, wherein the elements in the similarity measurement matrix are used to identify the weight value of each target class center for the proposed feature; and to perform a weighted summation of the target class center and the proposed feature based on the similarity measurement matrix to obtain the enhanced feature corresponding to the proposed feature. The determining unit is further configured to determine the target image features corresponding to the image to be detected based on the enhanced features; The detection unit is used to perform target detection on the image to be detected based on the target image features, and obtain the detection result.

12. The apparatus according to claim 11, characterized in that, The device further includes an input unit, which is used to input the image to be detected into a target detection model. The target detection model includes a feature extraction module, a class center module, a feature fusion module, and a recognition module. The class center module is composed of class centers in the class center dataset. The extraction unit is used to extract the proposed features from the image to be detected through the feature extraction module. The determining unit is configured to determine the target class center associated with the proposed feature through the class center module; The enhancement unit is used to fuse the target class center and the proposed feature through the feature fusion module to obtain the enhanced feature; The determining unit is further configured to determine the target image features corresponding to the image to be detected based on the fused features through the feature fusion module; The detection unit is used to perform target detection on the image to be detected by the recognition module based on the target image features, and obtain the detection result.

13. The apparatus according to claim 12, characterized in that, The apparatus further includes a construction unit, the construction unit being configured to construct the class center module, the construction unit comprising: During the training of the target detection model, the feature extraction module extracts features from the training sample images to obtain sample proposal features of objects identified by category labels. The category labels are used to identify the categories of objects included in the training sample images. The class center of the category to which the object belongs in the class center module is updated based on the sample proposal features of the object.

14. The apparatus according to claim 13, characterized in that, The building unit is also used for: If the number of class centers of the category to which the object belongs is zero, then the sample proposal feature of the object is used as the class center of the category to which the object belongs; If the number of class centers of the category to which the object belongs is not zero, then calculate the similarity between the sample proposal features of the object and the class centers of the category to which the object belongs; The class center of the category to which the object belongs in the class center module is updated based on the similarity.

15. The apparatus according to claim 14, characterized in that, The building unit is also used for: If the similarity is less than the similarity threshold, the sample proposal features of the object are added to the class center of the category to which the object belongs.

16. The apparatus according to claim 14, characterized in that, The building unit is also used for: If the similarity is less than the similarity threshold, determine the relationship between the number of class centers of the category to which the object belongs and the number threshold; Update the class center of the category to which the object belongs in the class center module according to the size relationship.

17. The apparatus according to claim 13, characterized in that, The building unit is also used for: Construct a target loss function based on the difference between the category labels and the obtained sample enhancement features; The class centers in the class center module are updated according to the target loss function to construct the class center module.

18. The apparatus according to any one of claims 11-17, characterized in that, Different class center datasets correspond to different environmental conditions, and the determining unit is used for: Obtain the class center dataset corresponding to the target environmental conditions; The target class center associated with the proposed feature is determined from the class center dataset corresponding to the target environmental conditions.

19. The apparatus according to any one of claims 11-17, characterized in that, The determining unit is used for: The enhanced features are concatenated with the proposed features to obtain the target image features.

20. The apparatus according to any one of claims 12-17, characterized in that, The device further includes a training unit, the training unit being configured to: Acquire training sample images under sample environment conditions, wherein the training sample images have category labels for the objects they include; The training sample images are input into an initial detection model, which includes an initial feature extraction module, an initial class center module, an initial feature fusion module, and an initial recognition module. The training sample images have category labels for the objects they contain. The initial feature extraction module extracts features from the training sample images to obtain sample proposal features; The initial class center module determines the target sample class center associated with the sample proposal features based on the similarity between the sample proposal features and the class centers in the class center dataset. The initial feature fusion module enhances the proposed features of the samples using the target sample class center to obtain enhanced sample features. The initial feature fusion module determines the target sample image features corresponding to the training sample image based on the sample enhancement features. The initial recognition module performs target detection on the image to be detected based on the features of the target sample image to obtain the sample detection result. The initial detection model is trained based on the category label and the sample detection results to obtain the target detection model.

21. A device for target detection, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1-10 according to the instructions in the program code.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the method according to any one of claims 1-10.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-10.

Images