Training method and device of target detection network, equipment and storage medium

Abstract

Embodiments of the present application disclose a target detection network training method, device and equipment and a medium, wherein the target detection network training method comprises: obtaining a predicted matching pair set and a predicted non-matching pair set based on the similarity between a pseudo-region visual feature corresponding to a training image and a text feature corresponding to N sample images; wherein the pseudo-region visual feature is obtained by inputting the training image into a target detection network to be trained to extract a visual feature; the training image is composed by splicing the N sample images; N is an integer greater than 1; and based on the predicted matching pair set and the predicted non-matching pair set, adjusting network parameters of the target detection network to be trained, so that the detection result output by the adjusted target detection network meets a convergence condition.

Description

Technical Field

[0001] This application relates to the field of computer vision, and more particularly to a training method, apparatus, device, and storage medium for an object detection network. Background Technology

[0002] In related technologies, during the image-level supervision process, object detection networks typically extract the representation information of the entire image and align it with relevant text information; or they parse the relevant text information into nouns and match the corresponding regions on the image as labels. This approach involves representing the entire image, making it impossible to effectively learn the representations of the internal layers of the image. Alternatively, methods based on pseudo-labels may suffer from a large number of incorrect labels / mismatches. Summary of the Invention

[0003] This application provides a training technique for an object detection network.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for training an object detection network, the method comprising:

[0006] Based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images, a set of predicted matching pairs and a set of predicted non-matching pairs are obtained; wherein, the pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is a positive integer greater than 1;

[0007] Based on the predicted matching pair set and the predicted non-matching pair set, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition.

[0008] This application provides a training apparatus for an object detection network, the apparatus comprising:

[0009] The prediction module is used to obtain a set of predicted matching pairs and a set of predicted non-matching pairs based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images; wherein, the pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is a positive integer greater than 1.

[0010] The adjustment module is used to adjust the network parameters of the target detection network to be trained based on the predicted matching pair set and the predicted non-matching pair set, so that the detection results output by the adjusted target detection network meet the convergence condition.

[0011] Accordingly, this application also provides a computer device, which includes a memory and a processor. The memory stores computer-executable instructions, and the processor can implement the above-described training method for the target detection network when it executes the computer-executable instructions in the memory.

[0012] This application embodiment also provides a computer storage medium storing computer-executable instructions, which, when executed, can implement the above-described training method for the target detection network.

[0013] This application provides a training method, apparatus, device, and storage medium for an object detection network. First, based on the similarity between the pseudo-region visual features corresponding to a training image and the text features corresponding to N sample images, a predicted matching pair set and a predicted non-matching pair set are obtained. The pseudo-region visual features are obtained by inputting the training image into the object detection network to be trained for visual feature extraction. The training image is constructed by concatenating the N sample images, where N is a positive integer greater than 1. Then, based on the predicted matching pair set and the predicted non-matching pair set, the network parameters of the object detection network to be trained are adjusted so that the detection results output by the adjusted network satisfy the convergence condition. Thus, by determining the region-text pairs corresponding to the image, i.e., the predicted matching pair set and the predicted non-matching pair set, through the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to the N sample images, the region-text pairs corresponding to the image can be determined. This enables the learning of the region-level representation of the image without data annotation during the supervised process, thereby obtaining an object detection network with high detection performance.

[0014] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0016] Figure 1A flowchart illustrating the training method for the first object detection network provided in this application embodiment;

[0017] Figure 2 A flowchart illustrating the training method for the second object detection network provided in this application embodiment;

[0018] Figure 3 This is a flowchart illustrating the image stitching method implemented in the training method of the target detection network provided in this application embodiment;

[0019] Figure 4 A schematic diagram illustrating the framework for image stitching using the training method of the object detection network provided in the embodiments of this application;

[0020] Figure 5 A schematic diagram illustrating image-text pair matching using the target detection network training method provided in the embodiments of this application;

[0021] Figure 6(a) shows a schematic diagram of the matching between relevant visual features and text features during the training process of an object detection network in a related technology;

[0022] Figure 6(b) illustrates the matching between relevant visual features and text features during the training process of another object detection network in related technologies;

[0023] Figure 6(c) illustrates a schematic diagram of the matching between relevant visual features and text features during the training process of the object detection network provided in the embodiments of this application;

[0024] Figure 7 A schematic diagram of the composition structure of a training device for an object detection network provided in an embodiment of this application;

[0025] Figure 8 This is a schematic diagram of the composition structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the invention will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate the embodiments of this application, but are not intended to limit the scope of the embodiments of this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application belong. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of embodiments of this application.

[0030] The following describes an exemplary application of the training apparatus for the object detection network provided in the embodiments of this application. The apparatus provided in the embodiments of this application can be implemented as a computer device with image acquisition capabilities, such as a laptop, tablet, desktop computer, camera, mobile device (e.g., personal digital assistant, dedicated messaging device, portable gaming device), and other various types of user terminals, or it can be implemented as a server. The functions implemented by this method can be achieved by the processor in the computer device calling program code. Of course, the program code can be stored in the computer storage medium. Therefore, the computer device includes at least a processor and a storage medium.

[0031] This application provides a method for training an object detection network, such as... Figure 1 The diagram shown is a flowchart illustrating the training method for the first object detection network provided in this application embodiment; combined with... Figure 1 The steps shown are explained below:

[0032] Step S101: Based on the similarity between the visual features of the pseudo-regions corresponding to the training images and the text features corresponding to the N sample images, a set of predicted matching pairs and a set of predicted non-matching pairs are obtained.

[0033] The pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is an integer greater than 1.

[0034] The sample images can be images collected in any scene. At the same time, the content between any two sample images in the N sample images can be the same, partially the same, or completely different.

[0035] The training image can be obtained by stitching N sample images according to a preset stitching rule and then performing at least one of the following processing methods: translation, rotation, scaling.

[0036] In some embodiments, firstly, a training image composed of N sample images is input into the target detection network to be trained for visual feature extraction, thereby obtaining the visual features corresponding to the training image. Then, based on the premise that the training image is composed of N sample images, partial region feature extraction is performed on the visual features of the image, that is, each sample image in the N sample images is regarded as a corresponding pseudo region in the training image, and then the visual features corresponding to the pseudo region are extracted, i.e., pseudo region visual features; here, the number of pseudo region visual features is the same as the number of N sample images.

[0037] It should be noted that the object detection network to be trained can be any open-set object detection network to be trained, such as: Open-vocabulary object detection (OVD), Bag of Object Regions Network (BARON), etc.

[0038] In some embodiments, the predicted matching pairs in the predicted matching pair set may be: predicted matching pairs composed of pseudo-region visual features with a similarity greater than a first value and text features of the sample image; meanwhile, the predicted non-matching pairs in the predicted non-matching pair set may be: predicted non-matching pairs composed of pseudo-region visual features with a similarity less than or equal to a second value and text features of the sample image; here, the first value is greater than the second value; meanwhile, the number of predicted matching pairs included in the predicted matching pair set and the number of predicted non-matching pairs included in the predicted non-matching pair set may be the same or different.

[0039] By using relevant numerical values, the related image-text pairs composed of pseudo-region visual features and text features are classified, thereby dividing them into a predicted matching pair set and a predicted non-matching pair set. This classification process is efficient and concise. That is, the above step S101 can be implemented through the following process:

[0040] Determine the predicted matching pairs composed of pseudo-region visual features and text features with similarity greater than a first value to obtain the predicted matching pair set;

[0041] Determine the predicted matching pairs composed of pseudo-region visual features and text features whose similarity is less than or equal to the second value, and obtain the set of predicted non-matching pairs;

[0042] Wherein, the first value is greater than or equal to the second value.

[0043] Here, the first value can be any value between (x, 1), where x can be any value greater than 0 and less than 1, and the second value can be any value between (0, y), where y is less than or equal to x.

[0044] Step S102: Based on the predicted matching pair set and the predicted non-matching pair set, adjust the network parameters of the target detection network to be trained so that the detection results output by the adjusted target detection network meet the convergence condition.

[0045] In some embodiments, the corresponding loss can be determined based on the predicted matching pair set and the predicted non-matching pair set. Then, based on the obtained loss, the network parameters of the target detection network to be trained are adjusted so that the loss of the detection result output by the adjusted, i.e., trained, target detection network converges, i.e., the convergence condition is met.

[0046] Here, based on the matching relationship between N sample images and the text features corresponding to the N sample images, a first loss set is determined to supervise the similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set, and a second loss set is determined to supervise the difference between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set. Then, based on the first loss set and the second loss set, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition.

[0047] It should be noted that a combined pseudo-region set corresponding to the training image can be further obtained. Here, the combined pseudo-region set can be constructed by stitching together N sample images to form the training image. The pseudo-regions corresponding to the N sample images in the training image are combined according to a preset rule to obtain the combined pseudo-region set. Then, based on the combined pseudo-region set, the combined pseudo-region visual feature set corresponding to the training image and the combined text feature set corresponding to the N sample images are determined. Based on the similarity between the combined pseudo-region visual features in the combined pseudo-region visual feature set and the combined text features in the combined text feature set, a predicted set of combined matching pairs and a predicted set of combined non-matching pairs are obtained. Based on the predicted matching pairs, predicted non-matching pairs, predicted combined matching pairs, and predicted combined non-matching pairs, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition. In this way, the pseudo-regions corresponding to the training image can be enriched, thereby making the performance of the adjusted target detection network obtained based on the visual features and text feature pairs corresponding to the pseudo-regions better.

[0048] The target detection network training method provided in this application firstly obtains a predicted matching pair set and a predicted non-matching pair set based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images. The pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction. The training image is composed of the N sample images, where N is a positive integer greater than 1. Then, based on the predicted matching pair set and the predicted non-matching pair set, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition. Thus, by determining the region-text pairs corresponding to the image, i.e., the predicted matching pair set and the predicted non-matching pair set, through the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to the N sample images, the region-text pairs corresponding to the image can be determined. This enables the learning of the region-level representation of the image without data annotation during the supervised process, thereby obtaining a target detection network with high detection performance.

[0049] In some embodiments, the pseudo-regions corresponding to N sample images in the training image are combined to obtain a set of combined pseudo-regions corresponding to the training image. Then, based on this set of combined pseudo-regions, the visual features of the pseudo-regions corresponding to the training image, and the text features corresponding to the N sample images, a set of predicted combined matching pairs and a set of predicted combined non-matching pairs are obtained. Thus, by obtaining a richer set of pseudo-regions corresponding to the training images, the network parameters of the target detection network to be trained can be adjusted based on a larger set of region-text pairs corresponding to the images, thereby improving the detection performance of the adjusted target detection network. That is, before executing step S102 provided in the above embodiments, the training method for the target detection network provided in this application can also execute the following steps S201 and S202, such as... Figure 2 The diagram shown is a flowchart illustrating the training method for the second object detection network provided in this application embodiment. Figure 1 and Figure 2 The steps shown are explained below:

[0050] Step S201: Combine the pseudo regions corresponding to the N sample images in the training image to obtain the combined pseudo region set corresponding to the training image.

[0051] Combine the pseudo-regions corresponding to the N sample images in the training image; here, it can be based on the principle of adjacency, according to the number of pseudo-regions, to combine them in pairs, or to combine two or more, etc.

[0052] In some possible implementations, based on the adjacent combination principle and the regular rectangle combination principle, the pseudo regions corresponding to the N sample images in the training image are further combined to obtain a set of combined pseudo regions. In this way, based on obtaining multiple regular rectangle pseudo regions efficiently and conveniently, the number of pseudo regions in the training image can be enriched. That is, step S201 provided in the above embodiment can be implemented through the following process:

[0053] Based on the principles of adjacent combination and regular rectangle combination, the pseudo regions corresponding to the N sample images in the training image are combined to obtain the set of combined pseudo regions.

[0054] In some embodiments, the adjacent combination principle is used, which means that the pseudo regions corresponding to N sample images in the training image are combined based on the adjacent principle, and the shape of the combined pseudo region is required to be a regular rectangle; wherein, the number of combined pseudo regions in the set of combined pseudo regions is related to the number of N; here, the combination of multiple adjacent pseudo regions can be 2, 3 or more, etc., and the maximum value of the number is less than N.

[0055] For example, if the training image is obtained by stitching 4 sample images together in a 2*2 pattern, then the training image corresponds to four pseudo-regions (each pseudo-region corresponds to one sample image). Then, these four pseudo-regions (top left pseudo-region A, bottom left pseudo-region B, top right pseudo-region C, and bottom right pseudo-region D) are combined according to the adjacent combination principle and the regular rectangle combination principle to obtain 4 combined pseudo-regions (the combined pseudo-regions formed by pseudo-regions A and B, the combined pseudo-regions formed by pseudo-regions C and D, the combined pseudo-regions formed by pseudo-regions A and C, and the combined pseudo-regions formed by pseudo-regions B and D). That is, the set of combined pseudo-regions includes 4 combined pseudo-regions.

[0056] Step S202: Based on the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images, and the text features corresponding to the N sample images, a predicted set of combined matching pairs and a predicted set of combined non-matching pairs are obtained.

[0057] Given a set of combined pseudo-regions corresponding to the training images, the visual features of the pseudo-regions corresponding to the training images are further combined based on the pseudo-regions in the set of combined pseudo-regions to obtain a set of combined pseudo-region visual features. Additionally, the text features corresponding to the N sample images are combined to obtain a set of combined text features.

[0058] In some possible implementations, firstly, based on the combined pseudo-regions in the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images are combined to obtain a combined pseudo-region visual feature set, and the text features corresponding to N sample images are combined to obtain a combined text feature set; then, based on the similarity between the combined pseudo-region visual features in the pseudo-region visual feature set and the combined text features in the combined text feature set, a predicted combined matching pair set and a predicted combined non-matching pair set are obtained; thus, a predicted combined matching pair set and a predicted combined non-matching pair set matching the combined pseudo-region set are obtained, which can provide richer training parameters for subsequent training, so as to achieve better detection performance of the adjusted target detection network obtained by training the target detection network to be trained. That is, step S202 provided in the above embodiment can be implemented by the following steps S2021 and S2022 (not shown in the figure):

[0059] Step S2021: Based on the combined pseudo-regions in the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images are combined to obtain a combined pseudo-region visual feature set, and the text features corresponding to the N sample images are combined to obtain a combined text feature set.

[0060] Based on the combination rules corresponding to the combined pseudo-regions in the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images are combined to obtain the combined pseudo-region visual feature set, and the text features corresponding to the N sample images are combined to obtain the combined text feature set.

[0061] The number of combined pseudo-regions in the combined pseudo-region set is equal to the number of combined pseudo-region visual features in the combined pseudo-region visual feature set and the number of combined text features in the combined text feature set.

[0062] Step S2022: Based on the similarity between the combined pseudo-region visual features in the combined pseudo-region visual feature set and the combined text features in the combined text feature set, a predicted set of combined matching pairs and a predicted set of combined non-matching pairs are obtained.

[0063] In some embodiments, the similarity between each combined pseudo-region visual feature in the combined pseudo-region visual feature set and all combined text features in the combined text feature set can be calculated. Based on the similarity value, predicted combined matching pairs composed of combined pseudo-region visual features with a similarity greater than a third value and predicted combined non-matching pairs composed of combined pseudo-region visual features with a similarity less than or equal to a fourth value are obtained, thereby obtaining corresponding sets of predicted combined matching pairs and sets of predicted combined non-matching pairs. Here, the third value is greater than or equal to the fourth value. The third value can be any value between (m, 1), where m can be any value greater than 0 and less than 1. The second value can be any value between (0, n), where n is less than or equal to m.

[0064] It should be noted that the relationship between n and m and x and y mentioned above can be determined according to actual needs. For example, n can be equal to y and m can be equal to x.

[0065] Accordingly, after obtaining the predicted combination matching pair set and the predicted combination non-matching pair set, the predicted matching pair set and the predicted non-matching pair set provided in the above embodiments can be merged to jointly adjust the network parameters of the target detection network to be trained, thereby making the adjusted target detection network perform better. That is, step S102 provided in the above embodiments can be implemented by the following step S203:

[0066] Step S203: Adjust the network parameters of the target detection network to be trained based on the predicted matching pair set, the predicted non-matching pair set, the predicted combined matching pair set, and the predicted combined non-matching pair set.

[0067] In some embodiments, a first loss set may be determined to supervise the similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set; a second loss set may be determined to supervise the similarity or difference between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set; a third loss set may be determined to supervise the similarity between the combined pseudo-region visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set; and the similarity between the combined pseudo-region visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set may be determined. A fourth loss set is used for supervision based on differences. Then, based on the first, second, third, and fourth loss sets, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition. Here, the first, second, third, and fourth loss sets can be fused, and the network parameters of the target detection network to be trained can be adjusted based on the fused loss. In this fusion, the first, second, third, and fourth loss sets can be assigned their respective weights, and then the above four loss sets can be fused based on these weights.

[0068] Here, the network parameters of the target detection network to be trained can refer to some weight values ​​of the target detection network to be trained, or the number of convolutions obtained by the target detection network to be trained.

[0069] In some possible implementations, based on the first matching relationship between N sample images and the text features corresponding to the N sample images, a first loss set corresponding to the matching pair set, a second loss set corresponding to the non-matching pair set, a third loss set corresponding to the combined matching pair set, and a fourth loss set corresponding to the combined non-matching pair set are predicted. Then, based on these first, second, third, and fourth loss sets, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network satisfy the convergence condition. In this way, based on the first matching relationship between N sample images and the text features corresponding to the N sample images, the relevant losses are conveniently and efficiently determined, providing a parameter basis for subsequent training of the target detection network to be trained. That is, step S203 provided in the above embodiment can be implemented through the following steps S2031 and S2032 (not shown in the figure):

[0070] Step S2031: Based on the first matching relationship between the N sample images and the text features corresponding to the N sample images, determine a first loss set that supervises the similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set, and a second loss set that supervises the difference between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set.

[0071] In some embodiments, the first matching relationship between N sample images and the text features corresponding to N sample images includes: the matching degree between sample image 1 and the text features corresponding to sample image 1 is 1, the matching degree between sample image 1 and the text features corresponding to sample image 2 is 0, the matching degree between sample image 1 and the text features corresponding to sample image 3 is 0, the matching degree between sample image 2 and the text features corresponding to sample image 2 is 1, etc.

[0072] In some embodiments, based on the first matching relationship between N sample images and the text features corresponding to the N sample images, a first loss set is determined to supervise the similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set, and a second loss set is determined to supervise the difference between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set. Here, it can be supervising the similarity between similarity 1 (usually represented by 1) and similarity 2 (usually a value between 0 and 1) between the pseudo-region visual features and text features that make up the predicted matching pair in the first matching relationship, or supervising the similarity between similarity 3 (usually represented by 0) and similarity 4 (usually a value between 0 and 1) between the pseudo-region visual features and text features that make up the predicted non-matching pair in the first matching relationship, and similarity 4 (usually a value between 0 and 1) between the pseudo-region visual features and text features that make up the predicted non-matching pair.

[0073] It should be noted that the first matching relationship can be represented by a numerical value. That is, when there is a match, the corresponding numerical value can be 1, and when there is no match, the corresponding numerical value can be 0. The similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set can be represented by any numerical value between x and 1. Similarly, the similarity, i.e., the difference, between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set can be represented by any numerical value between 0 and y, where x is greater than or equal to y.

[0074] Step S2032: Based on the first matching relationship, determine a third loss set to supervise the similarity between the visual features of the combined regions and the combined text features that make up each combined matching pair in the predicted combined matching pair set, and a fourth loss set to supervise the difference between the visual features of the combined regions and the combined text features that make up each combined non-matching pair in the predicted combined non-matching pair set.

[0075] In some possible implementations, the first matching relationship is first analyzed to obtain the second matching relationship between the combined pseudo-regions in the combined pseudo-region set and the combined text features in the combined text feature set; then, based on the second matching relationship, the third loss set and the fourth loss set corresponding to the predicted combined matching pair set and the predicted combined non-matching pair set are determined. That is, step S2032 provided in the above embodiment can be implemented through the following process:

[0076] The first step is to analyze the first matching relationship to obtain a second matching relationship between the combined pseudo-regions in the combined pseudo-region set and the combined text features in the combined text feature set.

[0077] For example, when N is 4 in N sample images, the first matching relationship includes: the matching degree between sample image 1 and the text feature corresponding to sample image 1 is 1, the matching degree between sample image 1 and the text feature corresponding to sample image 2 is 0, the matching degree between sample image 1 and the text feature corresponding to sample image 3 is 0, the matching degree between sample image 1 and the text feature corresponding to sample image 4 is 0, the matching degree between sample image 2 and the text feature corresponding to sample image 2 is 1, etc.; and the second matching relationship is: the matching degree between combined pseudo-region 1 (sample image 1 and sample image 2) and: the matching degree between combined text feature 1 corresponding to sample image 1 and sample image 2 is 1, the matching degree between combined pseudo-region 2 (sample image 1 and sample image 2) and: the matching degree between combined text feature corresponding to sample image 1 and sample image 3 is 0.5, etc.

[0078] The second step involves determining, based on the second matching relationship, a third loss set to supervise the similarity between the combined visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set to supervise the difference between the combined visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set.

[0079] In some embodiments, each of the predicted combined matching pairs in the set of predicted combined matching pairs and each of the predicted combined non-matching pairs in the set of predicted combined non-matching pairs can be supervised based on the second matching relationship, thereby obtaining the corresponding third loss set and fourth loss set.

[0080] For the specific implementation method, please refer to the description of step S2031 above, which will not be repeated here.

[0081] Step S2033: Adjust the network parameters of the target detection network to be trained based on the first loss set, the second loss set, the third loss set, and the fourth loss set.

[0082] In some embodiments, the first loss set, the second loss set, the third loss set, and the fourth loss set can be fused to obtain the total loss. Then, the network parameters of the target detection network to be trained can be adjusted based on the total loss so that the loss of the detection result of the adjusted target detection network output satisfies the convergence condition. In this way, by training the target detection network to be trained, the detection effect of the entire network can be improved, thereby obtaining a target detection network with higher performance.

[0083] In some embodiments, when N is the square of M, and M is a positive integer greater than 1, N sample images can be stitched together in an M*M pattern to form a training image. In this way, each of the N sample images corresponds to a square region in the training image, making subsequent extraction of the corresponding pseudo-region visual features more convenient and accurate. Figure 3 The diagram shown illustrates the process of image stitching in the training method of the target detection network provided in this application embodiment. Figure 3 The steps shown are explained below:

[0084] Step S301: Based on a preset cropping principle, at least some of the sample images in the N sample images are cropped to obtain N images to be stitched together.

[0085] In some embodiments, the preset cropping principle can be a regular image cropping principle or an irregular image cropping principle. For example, when the preset cropping principle is a regular image cropping principle, it can be a rectangular image cropping principle or a square image cropping principle. Then, when the preset cropping principle is a square image cropping principle, at least some of the sample images in the N sample images are cropped, that is, N images to be stitched are obtained, and the length and width of the images to be stitched are equal, thus providing sample preparation for subsequent stitching of training images in an M*M manner.

[0086] It should be noted that cropping at least some of the sample images in the N sample images can be done by cropping each of the N sample images. That is, if none of the N sample images meet the image size or image rules corresponding to the preset cropping principle, then each of the N sample images needs to be cropped so that each of the N sample images meets the image size or image rules corresponding to the preset cropping principle. Alternatively, it can be done by cropping M sample images in the N sample images, where M is greater than 0 and less than N. That is, there are L sample images (M=NL) in the N sample images that meet the image size or image rules corresponding to the preset cropping principle. Then, only the M sample images in the N sample images whose image size or image rules do not meet the preset cropping principle are cropped, so that each of the N sample images also meets the image size or image rules corresponding to the preset cropping principle.

[0087] In some possible implementations, the length direction of at least a portion of the N sample images can be determined first, and then the N sample images can be cropped along the length direction of at least a portion of the sample images to obtain N square images to be stitched together. In this way, the main content of each sample image in the N sample images can be retained as much as possible. That is, step S301 provided in the above embodiment can be implemented by the following steps S3011 and S3012 (not shown in the figure):

[0088] Step S3011: Determine the length direction of at least a portion of the N sample images.

[0089] In some embodiments, the direction of the long side of each of the N sample images is determined as the length direction of each of the N sample images. Here, the long side of the sample image can be measured based on the length and width dimensions of the sample image, and its corresponding long side can be determined.

[0090] Step S3012: Based on the preset cropping principle, at least a portion of the sample images in the N sample images are cropped along the length direction of the at least a portion of the sample images to obtain the N images to be stitched together.

[0091] Step S302: The N images to be stitched together are stitched together to obtain the training image.

[0092] In some embodiments, N images to be stitched together can be stitched together in an M*M stitching pattern to obtain the training image.

[0093] In some possible implementations, N images to be stitched can first be scaled to a preset image size to obtain N intermediate images; then, the N intermediate images with identical image sizes can be stitched together to obtain the training image. This ensures that the sizes of the corresponding N pseudo-regions in the obtained training image are all the same, which facilitates the subsequent extraction of visual features of the pseudo-regions. That is, step S302 provided in the above embodiment can be implemented by the following steps S3021 and S3022 (not shown in the figure):

[0094] Step S3021: Scale the N images to be stitched to a preset image size to obtain N intermediate images.

[0095] Step S3022: The N intermediate images are stitched together to obtain the training image.

[0096] In some possible implementations, to enrich the diversity of the combined images, the image obtained by stitching together N sample images can be subjected to image processing such as translation and scaling to obtain the training image. That is, step S3032 provided in the above embodiment can be implemented through the following process:

[0097] The first step is to stitch the N intermediate images together to obtain an intermediate stitched image.

[0098] The second step is to perform image preprocessing on the intermediate stitched image to obtain the training image; wherein the image preprocessing includes at least one of the following: scaling, translation, and rotation.

[0099] Based on the above description, the training method for the target detection network provided in this application embodiment can be implemented by the following process for obtaining the visual features of the pseudo-regions corresponding to the training images:

[0100] The first step is to input the training image into the target detection network to be trained, extract visual features, and obtain the image visual features corresponding to the training image.

[0101] The second step involves extracting features from the visual features of the image based on the location information of the pseudo-region corresponding to each of the N sample images in the training image, and taking the average value to obtain the visual features of the pseudo-region corresponding to the training image.

[0102] Here, based on the training method of the target detection network provided in the embodiments of this application, after obtaining an adjusted target detection network whose output detection result satisfies the convergence condition, the adjusted target detection network can be applied to a specific image detection process, thereby enabling the detection results obtained in the image detection process to have higher performance and greater accuracy. That is, the training method of the target detection network provided in the embodiments of this application can also realize the following process:

[0103] The first step is to acquire the image to be detected;

[0104] The second step is to input the image to be detected into the adjusted target detection network for image detection and obtain the detection result.

[0105] Here, the image to be detected can be an image corresponding to any scene, and its image format and image content can be determined according to actual needs.

[0106] By inputting the image to be detected into the adjusted target detection network for image detection, information such as the category of relevant objects in the image can be obtained.

[0107] The training method of the above-mentioned object detection network will be described below with reference to a specific embodiment. However, it should be noted that this specific embodiment is only for better illustrating the embodiments of this application and does not constitute an improper limitation on the embodiments of this application.

[0108] The goal of open-set detection is to enable object detectors to detect object categories not seen during training. This is typically achieved by using image-level labeled data as supervision to perceive new categories. Related techniques often utilize the abundant image-text pairs available on the internet for object detection. Through image-level supervision, the rich semantic concepts within the text significantly enhance the perceptual capabilities of the object detector. In image-level supervision, existing methods typically extract the representation of the entire image and align it with relevant text; or parse the text into nouns and match potentially corresponding regions on the image as labels. Additionally, visual localization data is also used to enhance the capabilities of open-set object detectors.

[0109] In related technologies, because the entire image is represented, there are problems such as the object detector being unable to effectively learn the representation at the region level / object level, or there are pseudo-label-based methods that cannot avoid a large number of incorrect labels / incorrect matches. Furthermore, the amount of data used in visual localization tasks is much smaller than the amount of data in image-text pairs, making it impossible to use this type of data to train large-scale open-set object detectors.

[0110] Based on this, embodiments of this application provide a training method for an object detection network, which mainly utilizes existing large-scale image-text pair data, without the need for pseudo-labels or additional visual localization data, to train an open-set object detector at the region level; the main steps are as follows:

[0111] The first step is the combination of images, such as... Figure 4 The diagram shown illustrates a framework for image stitching using the target detection network training method provided in this application embodiment; as shown... Figure 4As shown in 401, four images to be processed are acquired respectively, so that the images are stitched together in a 2×2 manner. Before stitching, the images are cropped along the long side to obtain images with equal length and width; then as shown in... Figure 4 As shown in 402, the cropped image is scaled to a fixed size and then stitched together with other images to obtain the result shown in Figure 402. Figure 4 The training image shown in 403; wherein, the scaling operation can be, for example, adjusting the cropped image to the same size as the preset reference image based on the image size corresponding to the preset reference image. Furthermore, in Figure 4 In 403, to enrich the diversity of the combined images, the combined images can be randomly translated and / or scaled. This can be done by translating and / or scaling the combined images while retaining most of the content, or by randomly translating and / or scaling the combined images based on a preset image size.

[0112] The second step involves comparative learning. After obtaining the stitched image, the present invention will... Figure 4 The four images to be processed corresponding to 401 in the middle are considered as Figure 4 The pseudo region of the image shown in 403 is used for visual feature extraction by contrastive learning. This allows the network parameters of the target detection network to be trained to be adjusted, enabling the target detection network to learn region-level features. Here, the training method of the target detection network provided in this application embodiment can be applied to various existing open set detection paradigms, including the pre-training stage and the model fine-tuning stage. These existing paradigms will not be described in detail here.

[0113] like Figure 5 The diagram shown illustrates image-text pair matching using the target detection network training method provided in this application embodiment; Figure 5The image shows a training image formed by stitching together four images 501a, 501b, 501c, and 501d (correspondingly, 501a, 501b, 501c, and 501d can be regarded as pseudo-regions corresponding to the training image). Simultaneously, the visual features of the image composed of 502a, 502b, 502c, and 502d are obtained by inputting the training image into the target detection network to be trained and extracting visual features. Correspondingly, 502a, 502b, 502c, and 502d are the pseudo-region visual features corresponding to the images 501a, 501b, 501c, and 501d in the training image, respectively. 503a, 503b, 503c, and 503d are the text features corresponding to the text description information of multiple images 501a, 501b, 501c, and 501d, respectively. Based on... Figure 5 As shown, according to the preset image stitching rules, the visual features of the pseudo-regions corresponding to each image to be processed are determined as r. i And the text feature corresponding to each image to be processed is t. i The goal of contrastive learning is to pull the corresponding r closer to the target value. i and t i Push away the non-corresponding r i and t i That is: maxs(r i ,t i mins(r) i ,t j ), i≠j; here, s represents similarity. Here, as... Figure 5 As shown, the maximum value of i and j is 4.

[0114] Furthermore, to enrich the diversity of pseudo-regions corresponding to the training images, embodiments of this application may also include... Figure 5 The pseudo regions 501c and 501d shown are combined according to a preset rule to obtain a new pseudo region, namely a combined pseudo region; the visual features of this combined pseudo region are 502c and 502d, and the text features are 503c and 503d. The visual features 502c and 502d of the combined pseudo regions 501c and 501d are brought closer to the corresponding text features 503c and 503d. Furthermore, the visual features 502c and 502d of the combined pseudo regions are pushed away from the text features 503a and 503b that do not correspond to them, i.e., do not match.

[0115] It should be noted that during the training of the target detection network to be trained, the composition can be based on... Figure 5The similarity between the four unprocessed images 501a, 501b, 501c, and 501d of the training image 501 and the corresponding text features 503a, 503b, 503c, and 503d is determined. A supervised loss is applied to determine the similarity between the predicted images 502a, 502b, 502c, and 502d of the target detection network to be trained and the images 503a, 503b, 503c, and 503d. Based on this determined loss, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition.

[0116] As shown in Figures 6(a) and 6(b), these are schematic diagrams illustrating the matching between relevant visual features and text features during the training process of the object detection network in the related art. Figures 6(a) and 6(b) are schematic diagrams illustrating the matching of visual features and text features based on the relevant visual features extracted by the visual encoder and text encoder, respectively, during the training process of the Contrastive Language-Image Pre-training (CLIP) model in the related art. Figure 6(c) is a schematic diagram illustrating the matching of visual features and text features during the training process of the object detection network using the training method of the object detection network provided in the embodiments of this application.

[0117] Here, in Figure 6(a), four images are simultaneously input into a visual encoder for visual feature extraction, resulting in the overall visual features corresponding to each of the four images. The corresponding text descriptions for the four images are then input into a text encoder for text feature extraction, yielding their respective text features. During training, corresponding overall visual and text features are brought closer together, while discontinuous overall visual and text features are pushed away. In Figure 6(b), it is shown that when matching the visual features and corresponding text features of an image, based on label annotation, corresponding partial visual and text features are brought closer together during training, while discontinuous partial visual and text features are pushed away. In Figure 6(c), based on the target training method provided in this application embodiment, the corresponding pseudo-region visual and text features of the spliced ​​image are brought closer together during training, while discontinuous pseudo-region visual and text features are pushed away.

[0118] The target detection network training method provided in this application embodiment is the Contrastive Language-Image Mosaic (CLIM) approach. It first stitches together multiple images. In the resulting large image, each sub-image is treated as a pseudo-region. This allows the pseudo-regions and their corresponding text to form a data structure similar to visual localization. Furthermore, by training the similarity between the visual features of the pseudo-regions corresponding to the image and the text features of N sample images, corresponding image-text pairs are determined, i.e., the predicted matching pair set and the predicted non-matching pair set. This enables the learning of image region-level representations without data annotation during the supervised process, resulting in a target detection network with high detection performance.

[0119] Based on the foregoing embodiments, this application provides a training apparatus for an object detection network. Figure 7 This is a schematic diagram illustrating the structural composition of a training device for an object detection network provided in an embodiment of this application, as shown below. Figure 7 As shown, the training device 700 for the target detection network includes:

[0120] The prediction module 701 is used to obtain a set of predicted matching pairs and a set of predicted non-matching pairs based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images; wherein, the pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is an integer greater than 1.

[0121] The adjustment module 702 is used to adjust the network parameters of the target detection network to be trained based on the predicted matching pair set and the predicted non-matching pair set, so that the detection result output by the adjusted target detection network satisfies the convergence condition.

[0122] In some embodiments, the training apparatus 700 for the target detection network further includes: a combination module 703, configured to combine the pseudo regions corresponding to the N sample images in the training image to obtain a set of combined pseudo regions corresponding to the training image; a prediction module 701, configured to obtain a set of predicted combined matching pairs and a set of predicted combined non-matching pairs based on the set of combined pseudo regions, the visual features of the pseudo regions corresponding to the training image, and the text features corresponding to the N sample images; and an adjustment module 702, configured to adjust the network parameters of the target detection network to be trained based on the set of predicted matching pairs, the set of predicted non-matching pairs, the set of predicted combined matching pairs, and the set of predicted combined non-matching pairs.

[0123] In some embodiments, the combination module 703 is further configured to combine the pseudo regions corresponding to the N sample images in the training image based on the adjacent combination principle and the regular rectangle combination principle to obtain the combined pseudo region set.

[0124] In some embodiments, the prediction module 701 is further configured to combine the visual features of the pseudo-regions corresponding to the training images based on the combined pseudo-regions in the combined pseudo-region set to obtain a combined pseudo-region visual feature set, and to combine the text features corresponding to the N sample images to obtain a combined text feature set; and to obtain a predicted combined matching pair set and a predicted combined non-matching pair set based on the similarity between the combined pseudo-region visual features in the combined pseudo-region visual feature set and the combined text features in the combined text feature set.

[0125] In some embodiments, the adjustment module 702 is further configured to: determine a first loss set that supervises the similarity between the pseudo-region visual features and text features constituting each predicted matching pair in the predicted matching pair set, and a second loss set that supervises the difference between the pseudo-region visual features and text features constituting each predicted non-matching pair in the predicted non-matching pair set, based on a first matching relationship between the N sample images and the text features corresponding to the N sample images; determine a third loss set that supervises the similarity between the combined region visual features and combined text features constituting each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set that supervises the difference between the combined region visual features and combined text features constituting each predicted combined non-matching pair in the predicted combined non-matching pair set; and adjust the network parameters of the target detection network to be trained based on the first loss set, the second loss set, the third loss set, and the fourth loss set.

[0126] In some embodiments, the adjustment module 702 is further configured to analyze the first matching relationship to obtain a second matching relationship between the combined pseudo-regions in the combined pseudo-region set and the combined text features in the combined text feature set; based on the second matching relationship, to determine a third loss set that supervises the similarity between the combined region visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set that supervises the difference between the combined region visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set.

[0127] In some embodiments, N is the square of M, and M is a positive integer greater than 1. The training device 700 for the object detection network further includes: a stitching module 704, used to crop at least a portion of the sample images in the N sample images based on a preset cropping principle to obtain N images to be stitched together; and to stitch the N images to be stitched together to obtain the training image.

[0128] In some embodiments, the stitching module 704 is further configured to determine the length direction of at least a portion of the sample images in the N sample images; based on the preset cropping principle, the at least a portion of the sample images in the N sample images are cropped along the length direction of the at least a portion of the sample images to obtain the N images to be stitched together.

[0129] In some embodiments, the stitching module 704 is further configured to scale the N images to be stitched to a preset image size to obtain N intermediate images; and stitch the N intermediate images together to obtain the training image.

[0130] In some embodiments, the stitching module 704 is further configured to stitch the N intermediate images to obtain an intermediate stitched image; and to perform image preprocessing on the intermediate stitched image to obtain the training image; wherein the image preprocessing includes at least one of the following: scaling, translation, and rotation.

[0131] In some embodiments, the training device 700 for the target detection network further includes: an extraction module 705, configured to input the training image into the target detection network to be trained, extract visual features to obtain image visual features corresponding to the training image; and based on the location information of the pseudo-region corresponding to each of the N sample images in the training image, extract features from the image visual features and take the average value to obtain the pseudo-region visual features corresponding to the training image.

[0132] In some embodiments, the training device 700 for the target detection network further includes: a detection module 706, used to acquire an image to be detected; input the image to be detected into the adjusted target detection network for image detection, and obtain a detection result.

[0133] It should be noted that the descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0134] It should be noted that, in the embodiments of this application, if the above-mentioned target detection network training method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a smartphone, tablet computer, etc. with a camera) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0135] Based on the same technical concept, this application provides a computer device for implementing the training method of the target detection network described in the above method embodiments. Figure 8 This is a schematic diagram of the composition structure of a computer device provided in an embodiment of this application, such as... Figure 8 As shown, the computer device 800 includes: a processor 801, at least one communication bus, a communication interface 802, at least one external communication interface, and a memory 803. The communication interface 802 is configured to enable communication between these components. The communication interface 802 may include a display screen, and the external communication interface may include standard wired and wireless interfaces. The processor 801 is configured to execute a program in the memory to implement the target detection network training method provided in the above embodiments.

[0136] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the training method of the target detection network described in any of the above embodiments.

[0137] Accordingly, in this application embodiment, a chip is also provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is running, is used to implement the training method of the target detection network described in any of the above embodiments.

[0138] Accordingly, in this application embodiment, a computer program product is also provided, which, when executed by the processor of an electronic device, is used to implement the training method of the target detection network described in any of the above embodiments.

[0139] The descriptions of the training apparatus, computer equipment, and storage medium embodiments of the object detection network above are similar to the descriptions of the method embodiments above, and have similar technical descriptions and beneficial effects. Due to space limitations, please refer to the descriptions of the method embodiments above, and therefore, they will not be repeated here. For technical details not disclosed in the embodiments of the object detection network provided in this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0140] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence number of the above-described processes does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0142] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium, and when executed, it performs the steps of the above method embodiments. Alternatively, if the integrated unit of this application is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above descriptions are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an object detection network, characterized in that, The method includes: Based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images, a set of predicted matching pairs and a set of predicted non-matching pairs are obtained; wherein, the pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is an integer greater than 1. The pseudo regions corresponding to the N sample images in the training image are combined to obtain the combined pseudo region set corresponding to the training image. Based on the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images, and the text features corresponding to the N sample images, a set of predicted combined matching pairs and a set of predicted combined non-matching pairs are obtained. Based on the predicted matching pair set, the predicted non-matching pair set, the predicted combined matching pair set, and the predicted combined non-matching pair set, the network parameters of the target detection network to be trained are adjusted so that the detection results output by the adjusted target detection network meet the convergence condition.

2. The training method according to claim 1, characterized in that, The step of combining the pseudo-regions corresponding to the N sample images in the training image to obtain the combined pseudo-region set corresponding to the training image includes: Based on the adjacent combination principle and the regular rectangle combination principle, the pseudo regions corresponding to the N sample images in the training image are combined to obtain the combined pseudo region set.

3. The training method according to claim 2, characterized in that, The process of obtaining a predicted set of combined matching pairs and a predicted set of combined non-matching pairs based on the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images, and the text features corresponding to the N sample images includes: Based on the combined pseudo-regions in the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images are combined to obtain a combined pseudo-region visual feature set, and the text features corresponding to the N sample images are combined to obtain a combined text feature set. Based on the similarity between the combined pseudo-region visual features in the combined pseudo-region visual feature set and the combined text features in the combined text feature set, a predicted set of combined matching pairs and a predicted set of combined non-matching pairs are obtained.

4. The training method according to any one of claims 1 to 3, characterized in that, The step of adjusting the network parameters of the target detection network to be trained based on the predicted matching pair set, the predicted non-matching pair set, the predicted combined matching pair set, and the predicted combined non-matching pair set includes: Based on the first matching relationship between the N sample images and the text features corresponding to the N sample images, a first loss set is determined to supervise the similarity between the pseudo-region visual features and text features that make up each predicted matching pair in the predicted matching pair set, and a second loss set is determined to supervise the difference between the pseudo-region visual features and text features that make up each predicted non-matching pair in the predicted non-matching pair set. Based on the first matching relationship, a third loss set is determined to supervise the similarity between the combined region visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set is determined to supervise the difference between the combined region visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set. The network parameters of the target detection network to be trained are adjusted based on the first loss set, the second loss set, the third loss set, and the fourth loss set.

5. The training method according to claim 4, characterized in that, The step of determining a third loss set based on the first matching relationship, which supervises the similarity between the combined visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set that supervises the difference between the combined visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set, includes: Analyzing the first matching relationship yields a second matching relationship between the combined pseudo-regions in the combined pseudo-region set and the combined text features in the combined text feature set; Based on the second matching relationship, a third loss set is determined to supervise the similarity between the combined region visual features and combined text features that make up each predicted combined matching pair in the predicted combined matching pair set, and a fourth loss set is determined to supervise the difference between the combined region visual features and combined text features that make up each predicted combined non-matching pair in the predicted combined non-matching pair set.

6. The training method according to claim 1, characterized in that, The method further includes: N is the square of M, where M is an integer greater than 1; Based on a preset cropping principle, at least a portion of the N sample images are cropped to obtain N images to be stitched together. The N images to be stitched together are used to obtain the training image.

7. The training method according to claim 6, characterized in that, The process of cropping at least a portion of the N sample images based on a preset cropping principle to obtain N images to be stitched together includes: Determine the length direction of at least a portion of the N sample images; Based on the preset cropping principle, at least a portion of the N sample images are cropped along the length direction of the at least a portion of the sample images to obtain the N images to be stitched together.

8. The training method according to claim 6 or 7, characterized in that, The step of stitching the N images to be stitched together to obtain the training image includes: The N images to be stitched are scaled to a preset image size to obtain N intermediate images; The N intermediate images are stitched together to obtain the training image.

9. The training method according to claim 8, characterized in that, The step of stitching together the N intermediate images to obtain the training image includes: The N intermediate images are stitched together to obtain an intermediate stitched image; The intermediate stitched image is preprocessed to obtain the training image; wherein the image preprocessing includes at least one of the following: scaling, translation, and rotation.

10. The training method according to any one of claims 1 to 3, 5 to 7, and 9, characterized in that, The method further includes: The training image is input into the target detection network to be trained, and visual features are extracted to obtain the image visual features corresponding to the training image. Based on the location information of the pseudo-region corresponding to each of the N sample images in the training image, features are extracted from the visual features of the image and averaged to obtain the visual features of the pseudo-region corresponding to the training image.

11. The method according to any one of claims 1 to 3, 5 to 7, and 9, characterized in that, After adjusting the network parameters of the target detection network to be trained based on the predicted matching pair set and the predicted non-matching pair set, so that the detection results output by the adjusted target detection network satisfy the convergence condition, the method further includes: Acquire the image to be detected; The image to be detected is input into the adjusted target detection network for image detection, and the detection result is obtained.

12. A training device for an object detection network, characterized in that, The device includes: The prediction module is used to obtain a set of predicted matching pairs and a set of predicted non-matching pairs based on the similarity between the pseudo-region visual features corresponding to the training image and the text features corresponding to N sample images; wherein, the pseudo-region visual features are obtained by inputting the training image into the target detection network to be trained for visual feature extraction; the training image is composed of the N sample images; and N is an integer greater than 1. The combination module is used to combine the pseudo regions corresponding to the N sample images in the training image to obtain the combined pseudo region set corresponding to the training image; The prediction module is further configured to obtain a set of predicted combined matching pairs and a set of predicted combined non-matching pairs based on the combined pseudo-region set, the visual features of the pseudo-regions corresponding to the training images, and the text features corresponding to the N sample images. The adjustment module is used to adjust the network parameters of the target detection network to be trained based on the predicted matching pair set, the predicted non-matching pair set, the predicted combined matching pair set, and the predicted combined non-matching pair set, so that the detection results output by the adjusted target detection network meet the convergence condition.

13. A computer device comprising a memory and a processor, the memory storing computer-executable instructions, wherein the processor, when executing the computer-executable instructions in the memory, is capable of implementing the training method of the target detection network according to any one of claims 1 to 11.

14. A computer storage medium storing computer-executable instructions, which, when executed, enable the training method of the target detection network according to any one of claims 1 to 11.

Images