Image detection method, device, computer equipment and storage medium

Abstract

The present disclosure provides an image detection method, apparatus, computer equipment and storage medium, wherein the method comprises: obtaining an image to be detected, and determining an initial semantic feature map of the image to be detected; wherein the initial semantic feature map contains multiple marker features; performing at least one fusion process on the multiple marker features in the initial semantic feature map, and obtaining a target fusion feature map corresponding to the initial semantic feature map after each fusion process; and determining a semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one target fusion feature map, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image detection method, apparatus, computer equipment, and storage medium. Background Art

[0002] When processing images, such as keypoint detection and image recognition, it is usually necessary to obtain the image's semantic information, and then perform subsequent processing based on the image's semantic information. Therefore, the accuracy of this semantic information directly affects the subsequent processing results. Therefore, how to accurately extract the semantic information of an image has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The embodiments of the present disclosure at least provide an image detection method, apparatus, computer equipment, and storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides an image detection method, comprising:

[0005] Acquire an image to be detected, and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes a plurality of marking features;

[0006] Performing at least one fusion process on the multiple marker features in the initial semantic feature map, and obtaining a target fusion feature map corresponding to the initial semantic feature map after each fusion process;

[0007] Based on the initial semantic feature map and at least one of the target fusion feature maps, a semantic feature map corresponding to the image to be detected is determined, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

[0008] In the above method, after determining the initial semantic feature map of the image to be detected, multiple marking features in the initial semantic feature map can be fused at least once to obtain at least one target fusion feature map corresponding to the initial semantic feature map, thereby reducing the number of marking features while retaining feature information; then, based on the initial semantic feature map and at least one target fusion feature map, the semantic feature map corresponding to the image to be detected is determined.

[0009] Since the marker features can represent the semantic information in the initial semantic feature map, when multiple marker features in the initial semantic feature map are fused at least once, the marker features of the same semantic information can be adaptively fused in combination with the semantic information in the initial semantic feature map. The important areas in the image to be detected can contain finely divided semantic information, so that the important areas in the image to be detected can be expressed by more marker features, thereby improving the accuracy of the semantic expression of the image to be detected.

[0010] In addition, by combining the semantic information in the initial semantic feature map and performing at least one fusion process on the multiple marker features in the initial semantic feature map, the feature range of each marker feature can be adjusted in a targeted manner, so that more refined marker features can be used to characterize small areas, thereby improving the semantic expression of the details of the image to be detected.

[0011] In a possible implementation, determining the initial semantic feature map of the image to be detected includes:

[0012] Performing preliminary feature extraction on the image to be detected to obtain an initial feature map corresponding to the image to be detected;

[0013] Semantic feature extraction is performed on the initial feature map to obtain the initial semantic feature map.

[0014] In a possible implementation, the marking feature is composed of a semantic feature of at least one feature point in the initial semantic feature map;

[0015] Among them, the number of semantic features of feature points contained in different marking features is the same.

[0016] Using this method, the semantic features of the initial semantic feature map can be converted into multiple label features for representation, thereby ensuring that the initial semantic feature map contains sufficiently rich image feature information, and further extracting more comprehensive image semantic information.

[0017] In a possible implementation, the one-time fusion process includes:

[0018] Based on the feature similarity between multiple marker features in the target fusion feature map of the previous fusion process, clustering the multiple marker features in the target fusion feature map, fusing the multiple marker features belonging to the same cluster, and determining the marker feature corresponding to the cluster;

[0019] Perform feature fusion on the marker features corresponding to each cluster and the target fusion feature map to determine an intermediate fusion feature map;

[0020] Performing deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion processing;

[0021] Among them, the target fusion feature map in the first fusion process is the initial semantic feature map.

[0022] By using this method, marker features with high feature similarity can be divided into the same cluster, and feature fusion can be performed on marker features belonging to the same cluster, which reduces the amount of calculation in subsequent processing.

[0023] In one possible implementation, the method is processed by a pre-trained semantic detection network;

[0024] The step of fusing multiple marker features belonging to the same cluster to determine the marker features corresponding to the cluster includes:

[0025] Determine the weight corresponding to each marker feature; the weight is a parameter adjusted during the training process of the semantic detection network;

[0026] Based on the feature values of multiple marker features belonging to the same cluster and the weights corresponding to the marker features, the feature value of the fused marker feature corresponding to the cluster is determined.

[0027] By adopting this method, the proportion of important marker features can be strengthened and the proportion of other marker features can be weakened, so that the feature values of the fused marker features corresponding to the cluster can retain more semantic information of important marker features.

[0028] In a possible implementation, when the number of the target fusion feature maps is M, determining the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and the M target fusion feature maps includes:

[0029] Upsampling each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine a second feature map corresponding to the N+1th target fusion feature map, wherein when N+1=M, the first feature map corresponding to the Nth target fusion feature map is the Nth target fusion feature map, and N and M are positive integers;

[0030] The second feature map corresponding to the N+1th target fusion feature map is fused and semantic feature extracted with the Nth target fusion feature map after full connection processing to obtain a first feature map corresponding to the Nth target fusion feature map; wherein, when N=1, the Nth target fusion feature map is the initial semantic feature map;

[0031] The semantic feature map corresponding to the image to be detected is determined based on the obtained first feature map corresponding to the initial semantic feature map.

[0032] Using this method, the initial semantic feature map can be restored to the size of the image to be detected, and during the restoration process, the semantic information of at least one target fusion feature map is fused, which can retain more semantic information, making the semantic information of the final semantic feature map more comprehensive and accurate.

[0033] In a possible implementation, upsampling each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine the second feature map corresponding to the N+1th target fusion feature map includes:

[0034] According to the clustering result in the process of performing fusion processing to obtain the N+1th target fusion feature map, each label feature in the first feature map corresponding to the N+1th target fusion feature map is up-sampled to determine the second feature map corresponding to the N+1th target fusion feature map.

[0035] By adopting this method, the corresponding position of each marker feature in the first feature map in the second feature map can be accurately determined, thereby making the semantic information of each marker feature in the final semantic feature map more accurate.

[0036] In a possible implementation, after determining the semantic feature map corresponding to the image to be detected, the method further includes:

[0037] Based on the semantic feature map corresponding to the image to be detected, position information of the key points of the image to be detected is determined.

[0038] In a second aspect, an embodiment of the present disclosure further provides an image detection device, comprising:

[0039] An acquisition module is used to acquire an image to be detected and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes multiple marking features;

[0040] A fusion module is used to perform at least one fusion process on the multiple marking features in the initial semantic feature map, and obtain a target fusion feature map corresponding to the initial semantic feature map after each fusion process;

[0041] A determination module is used to determine the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one target fusion feature map, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

[0042] In a possible implementation, the acquisition module, when determining the initial semantic feature map of the image to be detected, is configured to:

[0043] Performing preliminary feature extraction on the image to be detected to obtain an initial feature map corresponding to the image to be detected;

[0044] Semantic feature extraction is performed on the initial feature map to obtain the initial semantic feature map.

[0045] In a possible implementation, the marking feature is composed of a semantic feature of at least one feature point in the initial semantic feature map;

[0046] Among them, the number of semantic features of feature points contained in different marking features is the same.

[0047] In a possible implementation, the fusion module, when performing a fusion process, is configured to:

[0048] Based on the feature similarity between multiple marker features in the target fusion feature map of the previous fusion process, clustering the multiple marker features in the target fusion feature map, fusing the multiple marker features belonging to the same cluster, and determining the marker feature corresponding to the cluster;

[0049] Perform feature fusion on the marker features corresponding to each cluster and the target fusion feature map to determine an intermediate fusion feature map;

[0050] Performing deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion processing;

[0051] Among them, the target fusion feature map in the first fusion process is the initial semantic feature map.

[0052] In one possible implementation, the method is processed by a pre-trained semantic detection network;

[0053] The fusion module, when fusing multiple marker features belonging to the same cluster to determine the marker feature corresponding to the cluster, is used to:

[0054] Determine the weight corresponding to each marker feature; the weight is a parameter adjusted during the training process of the semantic detection network;

[0055] Based on the feature values of multiple marker features belonging to the same cluster and the weights corresponding to the marker features, the feature value of the fused marker feature corresponding to the cluster is determined.

[0056] In a possible implementation, when the number of the target fusion feature maps is M, the determination module is used to:

[0057] Upsampling each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine a second feature map corresponding to the N+1th target fusion feature map, wherein when N+1=M, the first feature map corresponding to the Nth target fusion feature map is the Nth target fusion feature map, and N and M are positive integers;

[0058] The second feature map corresponding to the N+1th target fusion feature map is fused and semantic feature extracted with the Nth target fusion feature map after full connection processing to obtain a first feature map corresponding to the Nth target fusion feature map; wherein, when N=1, the Nth target fusion feature map is the initial semantic feature map;

[0059] The semantic feature map corresponding to the image to be detected is determined based on the obtained first feature map corresponding to the initial semantic feature map.

[0060] In a possible implementation, the determination module, when performing upsampling processing on each marker feature in the first feature map corresponding to the N+1th target fused feature map to determine the second feature map corresponding to the N+1th target fused feature map, is used to:

[0061] According to the clustering result in the process of performing fusion processing to obtain the N+1th target fusion feature map, each label feature in the first feature map corresponding to the N+1th target fusion feature map is up-sampled to determine the second feature map corresponding to the N+1th target fusion feature map.

[0062] In a possible implementation, after determining the semantic feature map corresponding to the image to be detected, the determination module is further configured to:

[0063] Based on the semantic feature map corresponding to the image to be detected, position information of the key points of the image to be detected is determined.

[0064] In a third aspect, an embodiment of the present disclosure further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0065] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0066] For a description of the effects of the above-mentioned image detection device, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned image detection method, which will not be repeated here.

[0067] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure.

[0068] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0071] Figure 1 A flowchart of an image detection method provided by an embodiment of the present disclosure is shown;

[0072] Figure 2a A schematic diagram of determining an initial semantic feature graph provided by an embodiment of the present disclosure is shown;

[0073] Figure 2b A schematic diagram of a semantic extraction module provided by an embodiment of the present disclosure is shown;

[0074] Figure 3 A schematic diagram of an initial feature map after division provided by an embodiment of the present disclosure is shown;

[0075] Figure 4 A flowchart of a specific method for fusion processing in the image detection method provided by an embodiment of the present disclosure is shown;

[0076] Figure 5a A schematic diagram of a fusion processing process provided by an embodiment of the present disclosure is shown;

[0077] Figure 5b A schematic diagram showing the internal structure of a CTM provided by an embodiment of the present disclosure;

[0078] Figure 6 A flowchart showing a specific method for determining a semantic feature map corresponding to an image to be detected in the image detection method provided by an embodiment of the present disclosure is shown;

[0079] Figure 7aA schematic diagram of determining the fusion position and fusion quantity of a marker feature provided by an embodiment of the present disclosure is shown;

[0080] Figure 7b A schematic diagram of a marking feature replication provided by an embodiment of the present disclosure is shown;

[0081] Figure 8 A schematic diagram illustrating a process of determining a first feature map corresponding to an initial semantic feature map provided by an embodiment of the present disclosure is shown;

[0082] Figure 9 A flowchart of another image detection method provided by an embodiment of the present disclosure is shown;

[0083] Figure 10 A schematic diagram of the architecture of an image detection device provided by an embodiment of the present disclosure is shown;

[0084] Figure 11 A schematic structural diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0086] Human body detection and analysis tasks are important technical means in the fields of computer vision and intelligent video surveillance (such as human key point detection, facial key point detection, three-dimensional human body reconstruction, etc.). They also have important applications in many fields such as behavior recognition, virtual reality, and augmented reality.

[0087] In human detection and analysis tasks, when processing images, it is often necessary to obtain semantic information about the image and then perform subsequent processing based on this information. Transformers, as an emerging neural network architecture, are widely used in this field.

[0088] The Transformer network generally divides the image into multiple grid areas of the same shape and size, represents each grid area with a visual identifier (Vision token), and then uses the Vision token to extract image features.

[0089] However, if you want to obtain more accurate semantic information through this method, the grid area needs to be small enough, which may result in a large amount of data. If you want to increase the calculation speed, you need to divide the grid area into larger areas. The larger grid area may contain multiple semantic information. For example, a grid may contain both the body part of a person and the background part. It is difficult to reflect the difference between the body part of a person and the background part, or the difference between different parts of the human body. The semantic information obtained is less accurate.

[0090] Based on the above research, the present disclosure provides an image detection method, apparatus, computer equipment and storage medium, which can, after determining the initial semantic feature map of the image to be detected, perform at least one fusion process on multiple marking features in the initial semantic feature map to obtain at least one target fusion feature map corresponding to the initial semantic feature map, thereby reducing the number of marking features while retaining feature information; and then determine the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one target fusion feature map.

[0091] Since the marker features can represent the semantic information in the initial semantic feature map, when multiple marker features in the initial semantic feature map are fused at least once, the marker features of the same semantic information can be adaptively fused in combination with the semantic information in the initial semantic feature map. The important areas in the image to be detected can contain finely divided semantic information, so that the important areas in the image to be detected can be expressed by more marker features, thereby improving the accuracy of the semantic expression of the image to be detected.

[0092] In addition, by combining the semantic information in the initial semantic feature map and performing at least one fusion process on the multiple marker features in the initial semantic feature map, the feature range of each marker feature can be adjusted in a targeted manner, so that more refined marker features can be used to characterize small areas, thereby improving the semantic expression of the details of the image to be detected.

[0093] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0094] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0095] To facilitate understanding of this embodiment, we first provide a detailed introduction to an image detection method disclosed in this embodiment. The image detection method provided in this embodiment is generally executed by a computer device with certain computing capabilities, such as a terminal device, a server, or other processing device. The terminal device may include, for example, a personal computer. In some possible implementations, the image detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0096] See also Figure 1 FIG. 1 is a flowchart of an image detection method provided by an embodiment of the present disclosure, wherein the method includes steps 101 to 103, wherein:

[0097] Step 101: Acquire an image to be detected and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes multiple marker features;

[0098] Step 102: performing at least one fusion process on the multiple marker features in the initial semantic feature map, and obtaining a target fusion feature map corresponding to the initial semantic feature map after each fusion process;

[0099] Step 103: Determine a semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one of the target fusion feature maps, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

[0100] The following are detailed instructions for the above steps:

[0101] For step 101,

[0102] In one possible implementation, Figure 2a As shown, when determining the initial semantic feature map of the image to be detected, preliminary feature extraction can be performed on the image to be detected to obtain the initial feature map corresponding to the image to be detected, and then semantic feature extraction can be performed on the initial feature map to obtain the initial semantic feature map.

[0103] In some embodiments, the image detection method can be processed by a pre-trained semantic detection network. When performing preliminary feature extraction on the image to be detected, the image to be detected can be input into the preliminary feature extraction module of the semantic detection network to obtain an initial feature map corresponding to the image to be detected, and the initial feature map contains shallow features of the image to be detected; then, in order to extract the semantic features of the image to be detected, the initial feature map is input into the semantic extraction module of the semantic detection network. The semantic extraction module can be composed of multiple transformation modules (Transformer block). The structure of the exemplary semantic extraction module can be as follows Figure 2b shown.

[0104] The token can be understood as a partial feature in the initial semantic feature map. The multiple token features constitute the initial semantic feature map, so the token can be used to represent partial semantic information of the image to be detected. For example, the initial semantic feature map is an H*W*C feature map, where H and W represent the length and width of the feature map, and C represents the number of channels of the feature map. The token can be understood as a feature vector that constitutes the initial semantic feature map, for example, a 1*1*C dimensional feature vector.

[0105] In a possible implementation, the marking feature is composed of semantic features of at least one feature point in the initial semantic feature map; wherein different marking features contain the same number of semantic features of the feature points.

[0106] Exemplarily, the marking feature can be composed of the semantic feature of a feature point in the initial semantic feature map, that is, each feature point in the initial semantic feature map corresponds to a marking feature, and the feature value of the feature point is the value of the marking feature corresponding to the feature point.

[0107] Alternatively, in another example, the marker feature may be composed of semantic features of multiple feature points in the initial semantic feature map. When determining the marker feature, the multiple feature points may be divided into multiple regions, and then multiple marker features are generated based on the semantic features of the feature points in the multiple regions. Specifically, the following steps may be included:

[0108] Step A1: After obtaining the initial feature map corresponding to the image to be detected, the initial feature map may be divided according to preset sizes to determine multiple regions in the initial feature map;

[0109] Specifically, when the initial feature map is divided according to a preset size, the initial feature map may be divided into multiple regions equally according to a preset length and width, such as each region having a length of 3 pixels and a width of 2 pixels, or the initial feature map may be divided into multiple regions according to a preset number of regions, such as M×N, where M and N are both integers greater than or equal to 1. The initial feature map after division may be exemplarily as follows: Figure 3 shown.

[0110] Here, after the initial feature map is divided into the multiple regions, the position information of the multiple regions in the initial feature map is recorded.

[0111] Step A2: After semantic features are extracted from the initial feature map, the semantic features corresponding to all feature points contained in any region in the initial semantic feature map constitute a marking feature.

[0112] In some embodiments, based on the position information of the multiple regions in the initial feature map, the positions of the multiple regions in the initial semantic feature map can be determined, and then based on the positions of the multiple regions in the initial semantic feature map, the semantic features of the multiple regions in the initial semantic feature map can be determined, and finally the semantic features corresponding to the multiple regions can be extracted as the marking features.

[0113] Using this method, the semantic features of the initial semantic feature map can be converted into multiple label features for representation, thereby ensuring that the initial semantic feature map contains sufficiently rich image feature information, and further extracting more comprehensive image semantic information.

[0114] In one possible implementation, after performing preliminary feature extraction on the image to be detected and obtaining the initial feature map corresponding to the image to be detected, the initial feature map can be divided into multiple tokens according to the above method, and then semantic feature extraction can be performed to obtain an initial semantic feature map containing multiple tag features.

[0115] Alternatively, in another possible embodiment, after performing preliminary feature extraction on the image to be detected to obtain the initial feature map corresponding to the image to be detected, semantic feature extraction can be performed on the initial feature map to obtain the initial semantic feature map, and then the initial semantic feature map can be divided according to the above method to obtain an initial semantic feature map containing multiple marking features.

[0116] For step 102,

[0117] In a possible implementation, the Figure 4The method shown performs a fusion process, which specifically includes the following steps:

[0118] Step 401: clustering the multiple label features in the target fusion feature map of the previous fusion process based on the feature similarity between the multiple label features in the target fusion feature map of the previous fusion process; wherein the target fusion feature map of the first fusion process is the initial semantic feature map.

[0119] Among them, when clustering the multiple marker features in the target fusion feature map of the previous fusion process based on the feature similarity between the multiple marker features in the target fusion feature map of the previous fusion process, the cluster center feature can be first determined from the multiple marker features of the target fusion feature map of the previous fusion process, and then based on the feature similarity between the other marker features and the cluster center feature, the other marker features can be divided into clusters where different cluster center features are located.

[0120] Specifically, the clustering process may include the following steps:

[0121] Step B1: for any marker feature (hereinafter referred to as marker feature x i ), determine and mark the feature x i The k neighboring marking features with the closest feature distance between them, where k is an integer greater than or equal to 1, such as 5.

[0122] For example, the label feature x can be calculated by the K-Nearest Neighbor (KNN) algorithm. i The feature distances between the features and other marker features are sorted according to size, and the other marker features corresponding to the smallest k feature distances are used as the adjacent marker features.

[0123] Here, it should be noted that, with the marking feature x i The k other label features closest to the feature are not necessarily the same as the label feature x i Adjacent marker features.

[0124] Step B2: Based on the label feature x i and the adjacent marker features, determine the marker feature x i The local density of . For example, it can be calculated using the following formula:

[0125]

[0126] Among them, ρ i represents the local density of the i-th marker feature, k is the number of adjacent marker features, x i represents the i-th marker feature, x jRepresents the marker feature x i The corresponding neighboring label features, knn(x i ) represents the marker feature x i The set of corresponding neighboring marker features.

[0127] Step B3: Determine the ratio mark feature x from the adjacent mark features i The candidate marker features with high local density are determined from the candidate marker features. i The target marker feature with the closest feature distance is compared with the marker feature x i The characteristic distance between them is taken as the target distance.

[0128] The local density of the candidate marker feature is higher than the marker feature x i The local density of the candidate marker feature is greater than the marker feature x i More likely to become a cluster center feature, and the cluster center feature is similar to the label feature x i May become the same cluster, the cluster center feature should be the same as the label feature x i The features are similar, so when determining the cluster center features, the candidate marker features are combined with the marker features x i Feature distance.

[0129] Here, when the label feature x i When the local density is the largest, determine the marker feature x from the neighboring marker features. i The farthest marking feature with the largest feature distance between them, and mark the feature x i The characteristic distance between the feature and the farthest marker feature is used as the target distance.

[0130] Step B4: For each marker feature, after calculating the local density and target distance of each marker feature by the above method, multiply the local density of each marker feature by the target distance to obtain the clustering score of each marker feature, sort the clustering scores according to the scores, and use the preset number of marker features with the highest clustering scores as cluster center features.

[0131] Step B5: Divide the marker features other than the cluster center feature into the cluster center feature with the shortest feature distance to the cluster center feature, thereby obtaining multiple clusters.

[0132] Specifically, for any marker feature other than the cluster center feature, the feature distance between the marker feature and each cluster center feature may be calculated, and then the cluster to which the marker feature belongs may be determined.

[0133] Step 402: Fusing multiple marker features belonging to the same cluster to determine the marker feature corresponding to the cluster.

[0134] In one possible implementation, when multiple marker features belonging to the same cluster are fused to determine the marker feature corresponding to the cluster, the mean of the feature values of the multiple marker features belonging to the same cluster can be used as the feature value of the fused marker feature corresponding to the cluster.

[0135] Specifically, the characteristic values of the marker features of the same cluster can be added and divided by the number of marker features in the cluster to obtain the mean of the characteristic values of the marker features in the cluster, and the mean is used as the characteristic value of the fused marker feature corresponding to the cluster.

[0136] Furthermore, in a possible embodiment, the method can be executed by a pre-trained semantic detection network; when multiple marker features belonging to the same cluster are fused to determine the marker features corresponding to the cluster, the weights corresponding to each marker feature can be determined; the weights are parameters adjusted during the training of the semantic detection network; and then, based on the feature values of the multiple marker features belonging to the same cluster and the weights corresponding to each marker feature, the feature values of the fused marker features corresponding to the cluster are determined.

[0137] For example, when determining the characteristic value of the fused marker feature, it can be calculated using the following formula:

[0138]

[0139] Among them, y i represents the eigenvalue of the fused marker feature, i represents the cluster number, Ci represents the set of all marker feature numbers in the i-th cluster, j represents the marker feature number in the i-th cluster, x j represents the jth marker feature in the i-th cluster, and P represents the marker feature x j The weight of .

[0140] By adopting this method, the proportion of important marker features can be strengthened and the proportion of other marker features can be weakened, so that the feature values of the fused marker features corresponding to the cluster can retain more semantic information of important marker features.

[0141] Step 403: Fusing the marker features corresponding to each cluster with the target fusion feature map obtained in the previous fusion process to determine an intermediate fusion feature map.

[0142] After fusing multiple marker features belonging to the same cluster and determining the marker features corresponding to each cluster, in order to prevent the loss of detail information, the marker features corresponding to each cluster can be fused with the target fusion feature map of the previous fusion process.

[0143] Exemplarily, when performing feature fusion on the marker features corresponding to each class of clusters and the target fusion feature map of the previous fusion processing, the marker features corresponding to each class of clusters and the target fusion feature map of the previous fusion processing can be input into the Transformer block, and the Transformer block can input the intermediate fusion feature map.

[0144] Specifically, the Transformer block can extract the key feature information of each marker feature from the target fusion feature map of the previous fusion processing, calculate the feature similarity between the key feature information and the fused marker features at the corresponding position, multiply the feature value of each marker feature in the target fusion feature map of the previous fusion processing by the corresponding feature similarity, and add it to the marker features corresponding to each cluster to obtain the intermediate fusion feature map.

[0145] For example, it can be calculated by the following formula:

[0146]

[0147] Among them, Attention(Q,K,V) represents the result of multiplying the feature value of the marker feature by the corresponding feature similarity, Q represents the feature value of the fused marker feature, K represents the key feature information, V represents the feature value of the marker feature, d k Indicates the number of feature channels of the fused marker features.

[0148] Alternatively, in a possible implementation, after calculating the feature similarity between the key feature information and the fused marker features at the corresponding positions, the intermediate fused feature map may be calculated in combination with the weight of the target fused feature map of the previous fusion process.

[0149] For example, for any marker feature in the target fusion feature map of the previous fusion process, the following formula can be used for calculation:

[0150]

[0151] Among them, Attention(Q,K,V) represents the result of multiplying the feature value of the marker feature by the corresponding feature similarity, Q represents the feature value of the fused marker feature, K represents the key feature information, V represents the feature value of the marker feature, dk It represents the number of feature channels of the fused marker feature, and P represents the weight of the marker feature.

[0152] Step 404: perform deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion process.

[0153] Specifically, when performing deep semantic feature extraction on the intermediate fusion feature map, the intermediate fusion feature map can be input into a semantic extraction module, and the semantic extraction module can output the target fusion feature map. The target fusion feature map generates more feature channels through the deep semantic feature extraction to obtain deeper semantic information.

[0154] In summary, at least one fusion process is performed according to the above steps 401 to 404, and the latter fusion process is performed based on the previous fusion process. For example, in the case of three fusion processes, the fusion process can be as follows: Figure 5a As shown, the semantic extraction module is used to perform step 404 to extract semantic features, and cluster and fuse the tag features through the clustering-based token merge module (CTM), that is, to perform steps 401 to 403. The internal structure of the CTM is as follows: Figure 5b shown.

[0155] By adopting the method described in steps 401 to 404, marking features with high feature similarity can be divided into the same cluster, and feature fusion can be performed on the marking features belonging to the same cluster, thereby reducing the amount of calculation in subsequent processing.

[0156] For step 103,

[0157] In a possible implementation, when the number of the target fusion feature maps is M, when determining the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and the M target fusion feature maps, the following can be referred to: Figure 6 The method shown is as follows:

[0158] Step 601: Upsample each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine the second feature map corresponding to the N+1th target fusion feature map, wherein when N+1=M, the first feature map corresponding to the Nth target fusion feature map is the Nth target fusion feature map, and N and M are positive integers.

[0159] In a possible implementation, based on the clustering result obtained during the fusion process of the N+1th target fusion feature map, each marker feature in the first feature map corresponding to the N+1th target fusion feature map can be upsampled to determine the second feature map corresponding to the N+1th target fusion feature map.

[0160] The clustering result may include the fusion position and fusion quantity of the fused marker features fused into the N+1th target fusion feature map, and the feature value of each marker feature in the N+1th target fusion feature map.

[0161] Specifically, such as Figure 7a As shown, for any marker feature in the first feature map, the fusion position and fusion number of the marker feature in the Nth target fusion feature map can be determined, the fusion position is the position of the fused marker feature fused into the marker feature in the Nth target fusion feature map, and the fusion number is the number of the fused marker features. For example, if the marker feature A in the first feature map is fused by marker feature 1, marker feature 2 and marker feature 3 in the Nth target fusion feature map, then the fusion position of marker feature A in the Nth target fusion feature map is the position of marker feature 1, marker feature 2 and marker feature 3, and the fusion number is 3.

[0162] like Figure 7b As shown, after determining the fusion position and the fusion quantity, the marker feature can be copied according to the fusion quantity, and the copy result is used as the marker feature at the fusion position. Continuing with the above example, marker feature A is copied and used as the marker features at the positions of marker feature 1, marker feature 2, and marker feature 3. Similarly, the fusion position and fusion quantity of each marker feature in the first feature map are determined, and the copies are made to form the second feature map.

[0163] By adopting this method, the corresponding position of each marker feature in the first feature map in the second feature map can be accurately determined, thereby making the semantic information of each marker feature in the final semantic feature map more accurate.

[0164] Step 602: Fusing the second feature map corresponding to the N+1th target fusion feature map with the Nth target fusion feature map after full connection processing and performing semantic feature extraction processing to obtain a first feature map corresponding to the Nth target fusion feature map; wherein, when N=1, the Nth target fusion feature map is the initial semantic feature map;

[0165] Here, since deep semantic features are extracted during each fusion process, the sizes of different target fusion feature maps are different, and the upsampling in step 601 does not change the size of the target fusion feature map. Therefore, the Nth target fusion feature map needs to be fully connected to facilitate fusion.

[0166] Specifically, before fusion, in order to make the number of feature channels of the second feature map corresponding to the Nth target fusion feature map and the N+1th target fusion feature map the same, the Nth target fusion feature map can be first input into the linear layer for full connection processing to change the number of feature channels of the Nth target fusion feature map.

[0167] For example, if the number of feature channels of the second feature map corresponding to the N+1th target fusion feature map is 512, and the number of feature channels of the Nth target fusion feature map is 256, the Nth target fusion feature map can be fully connected to change the number of feature channels to 512.

[0168] Then, the feature values of the marking features at the corresponding positions of the second feature map and the Nth target fusion feature map are added respectively, and the intermediate feature map obtained after the addition is input into the Transformer block to obtain the first feature map corresponding to the Nth target fusion feature map.

[0169] Here, each time semantic feature extraction is performed, the semantic feature extraction module used may be composed of multiple layers of Transformer blocks, and the number of layers of the Transformer blocks of different semantic feature extraction modules may be the same or different.

[0170] For example, taking the number of target fusion feature maps as 4 (including the initial semantic feature map) as an example, the process of executing steps 601 to 602 to determine the first feature map corresponding to the initial semantic feature map can be as follows: Figure 8 shown.

[0171] By adopting the method described in steps 601 to 602, the initial semantic feature map can be restored to the size of the image to be detected, and in the restoration process, the semantic information of at least one target fusion feature map is fused, which can retain more semantic information, making the semantic information of the final semantic feature map more comprehensive and accurate.

[0172] Step 603: Determine a semantic feature map corresponding to the image to be detected based on the obtained first feature map corresponding to the initial semantic feature map.

[0173] After obtaining the first feature map corresponding to the initial semantic feature map, in order to ensure that the final semantic feature map is consistent with the size of the image to be detected, the first feature map corresponding to the initial semantic feature map can be reshaped.

[0174] The image detection method will be described below with reference to the accompanying drawings. Figure 9 FIG. 1 is a flowchart of an image detection method provided by an embodiment of the present disclosure, which mainly includes the following steps:

[0175] First, the image to be detected is input into the preliminary feature extraction module (such as the convolution layer) for preliminary feature extraction to obtain a preliminary feature map, and then the preliminary feature map is input into the semantic extraction module to obtain an initial semantic feature map;

[0176] Then the initial semantic feature map is input into the CTM, the marked features in the initial semantic feature map are clustered and fused, and the clustered feature map is input into the semantic extraction module to obtain the target fused feature map after a fusion process;

[0177] Then, based on the target fusion feature map and the initial semantic feature map obtained by multiple fusion processes, the semantic feature map corresponding to the image to be detected is determined.

[0178] Please refer to the description of the above embodiment for the specific execution steps, which will not be repeated here.

[0179] Here, the semantic categories contained in the semantic feature map are related to the application of the semantic feature map. For example, if the semantic feature map is used to perform facial key point recognition, the semantic categories contained in the semantic feature map may include the semantic categories of finely divided facial parts (such as the corners of the eyes, the corners of the mouth, the nose, the eyeballs, etc.), while the semantic information of the remaining parts may not be distinguished in detail. For example, the other parts of the human body may be unified as the same semantic information "torso". If the semantic feature map is used to perform gesture recognition, the semantic categories contained in the semantic features may include the semantic categories of finely divided hand parts (such as the fingertips, hand joints, wrists, palms, etc.), while the semantic information of the remaining parts may be distinguished as "head", "legs", etc.

[0180] In a possible implementation, after determining the semantic feature map corresponding to the image to be detected, position information of key points of the image to be detected may be determined based on the semantic feature map corresponding to the image to be detected.

[0181] Exemplarily, if human key point detection is to be performed on the image to be detected, the semantic feature map corresponding to the image to be detected can be obtained based on the above method, and then the position information of the human key points in the image to be detected can be determined based on the semantic feature map.

[0182] Specifically, the semantic feature map can be input into a pre-trained key point detection model. The key point detection model can determine the key points in the semantic feature map based on the semantic information of the semantic feature map, classify the key points, and then output the category of the key points, the location information of the key points, and the confidence that the key points belong to the category.

[0183] Similarly, facial key point detection, object key point detection, object recognition, etc. can also be performed based on the above method.

[0184] After determining the key point position information, further, state recognition can be performed based on the key point position information, such as gesture recognition based on the hand key point information, expression recognition based on the face key point information, action recognition based on the human body key point information, etc., and a three-dimensional model corresponding to the target object can be constructed based on the key point position information.

[0185] The image detection method provided by the embodiment of the present disclosure can, after determining the initial semantic feature map of the image to be detected, perform at least one fusion process on multiple marking features in the initial semantic feature map to obtain at least one target fusion feature map corresponding to the initial semantic feature map, thereby reducing the number of marking features while retaining feature information; and then determine the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one target fusion feature map.

[0186] Since the marker features can represent the semantic information in the initial semantic feature map, when multiple marker features in the initial semantic feature map are fused at least once, the marker features of the same semantic information can be adaptively fused in combination with the semantic information in the initial semantic feature map. The important areas in the image to be detected can contain finely divided semantic information, so that the important areas in the image to be detected can be expressed by more marker features, thereby improving the accuracy of the semantic expression of the image to be detected.

[0187] In addition, by combining the semantic information in the initial semantic feature map and performing at least one fusion process on the multiple marker features in the initial semantic feature map, the feature range of each marker feature can be adjusted in a targeted manner, so that more refined marker features can be used to characterize small areas, thereby improving the semantic expression of the details of the image to be detected.

[0188] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0189] Based on the same inventive concept, an image detection device corresponding to the image detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0190] Reference Figure 10 FIG. 1 is a schematic diagram of the architecture of an image detection device provided by an embodiment of the present disclosure, wherein the device includes: an acquisition module 1001, a fusion module 1002, and a determination module 1003; wherein,

[0191] The acquisition module 1001 is used to acquire an image to be detected and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes multiple marker features;

[0192] A fusion module 1002 is configured to perform at least one fusion process on the multiple marker features in the initial semantic feature map, and obtain a target fused feature map corresponding to the initial semantic feature map after each fusion process;

[0193] The determination module 1003 is used to determine the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one of the target fusion feature maps, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

[0194] In a possible implementation, the acquisition module 1001, when determining the initial semantic feature map of the image to be detected, is configured to:

[0195] Performing preliminary feature extraction on the image to be detected to obtain an initial feature map corresponding to the image to be detected;

[0196] Semantic feature extraction is performed on the initial feature map to obtain the initial semantic feature map.

[0197] In a possible implementation, the marking feature is composed of a semantic feature of at least one feature point in the initial semantic feature map;

[0198] Among them, the number of semantic features of feature points contained in different marking features is the same.

[0199] In a possible implementation, the fusion module 1002, when performing a fusion process, is configured to:

[0200] Based on the feature similarity between multiple marker features in the target fusion feature map of the previous fusion process, clustering the multiple marker features in the target fusion feature map, fusing the multiple marker features belonging to the same cluster, and determining the marker feature corresponding to the cluster;

[0201] Perform feature fusion on the marker features corresponding to each cluster and the target fusion feature map to determine an intermediate fusion feature map;

[0202] Performing deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion processing;

[0203] Among them, the target fusion feature map in the first fusion process is the initial semantic feature map.

[0204] In one possible implementation, the method is processed by a pre-trained semantic detection network;

[0205] The fusion module 1002 is used to fuse multiple marker features belonging to the same cluster to determine the marker feature corresponding to the cluster:

[0206] Determine the weight corresponding to each marker feature; the weight is a parameter adjusted during the training process of the semantic detection network;

[0207] Based on the feature values of multiple marker features belonging to the same cluster and the weights corresponding to the marker features, the feature value of the fused marker feature corresponding to the cluster is determined.

[0208] In a possible implementation, the determination module 1003, when the number of the target fusion feature maps is M, is used to determine the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and the M target fusion feature maps:

[0209] Upsampling each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine a second feature map corresponding to the N+1th target fusion feature map, wherein when N+1=M, the first feature map corresponding to the Nth target fusion feature map is the Nth target fusion feature map, and N and M are positive integers;

[0210] The second feature map corresponding to the N+1th target fusion feature map is fused and semantic feature extracted with the Nth target fusion feature map after full connection processing to obtain a first feature map corresponding to the Nth target fusion feature map; wherein, when N=1, the Nth target fusion feature map is the initial semantic feature map;

[0211] The semantic feature map corresponding to the image to be detected is determined based on the obtained first feature map corresponding to the initial semantic feature map.

[0212] In a possible implementation, the determination module 1003, when performing upsampling processing on each marker feature in the first feature map corresponding to the N+1th target fused feature map to determine the second feature map corresponding to the N+1th target fused feature map, is used to:

[0213] According to the clustering result in the process of performing fusion processing to obtain the N+1th target fusion feature map, each label feature in the first feature map corresponding to the N+1th target fusion feature map is up-sampled to determine the second feature map corresponding to the N+1th target fusion feature map.

[0214] In a possible implementation, after determining the semantic feature map corresponding to the image to be detected, the determining module 1003 is further configured to:

[0215] Based on the semantic feature map corresponding to the image to be detected, position information of the key points of the image to be detected is determined.

[0216] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0217] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 11 1 is a schematic diagram of the structure of a computer device 1100 provided in an embodiment of the present disclosure, including a processor 1101, a memory 1102, and a bus 1103. The memory 1102 is used to store execution instructions and includes a memory 11021 and an external memory 11022. The memory 11021 is also referred to as internal memory, and is used to temporarily store operation data in the processor 1101, as well as data exchanged with an external memory 11022 such as a hard disk. The processor 1101 exchanges data with the external memory 11022 via the memory 11021. When the computer device 1100 is running, the processor 1101 communicates with the memory 1102 via the bus 1103, so that the processor 1101 executes the following instructions:

[0218] Acquire an image to be detected, and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes a plurality of marking features;

[0219] Performing at least one fusion process on the multiple marker features in the initial semantic feature map, and obtaining a target fusion feature map corresponding to the initial semantic feature map after each fusion process;

[0220] Based on the initial semantic feature map and at least one of the target fusion feature maps, a semantic feature map corresponding to the image to be detected is determined, wherein the semantic feature map is used to determine the semantic information of each pixel point of the image to be detected.

[0221] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the image detection method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0222] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the image detection method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0223] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0224] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0225] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0226] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0227] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0228] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

[0229] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. An image detection method, characterized in that: include: Acquire an image to be detected, and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes a plurality of marking features; Performing at least one fusion process on the multiple marker features in the initial semantic feature map, and obtaining a target fusion feature map corresponding to the initial semantic feature map after each fusion process; Determine a semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one of the target fusion feature maps, wherein the semantic feature map is used to determine semantic information of each pixel point of the image to be detected; The one-time fusion process includes: Based on the feature similarity between multiple marker features in the target fusion feature map of the previous fusion process, clustering the multiple marker features in the target fusion feature map, fusing the multiple marker features belonging to the same cluster, and determining the marker feature corresponding to the cluster; Perform feature fusion on the marker features corresponding to each cluster and the target fusion feature map to determine an intermediate fusion feature map; Performing deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion processing; Among them, the target fusion feature map in the first fusion process is the initial semantic feature map.

2. The method according to claim 1, characterized in that Determining the initial semantic feature map of the image to be detected includes: Performing preliminary feature extraction on the image to be detected to obtain an initial feature map corresponding to the image to be detected; Semantic feature extraction is performed on the initial feature map to obtain the initial semantic feature map.

3. The method according to claim 1 or 2, characterized in that The marking feature is composed of the semantic feature of at least one feature point in the initial semantic feature map; Among them, the number of semantic features of feature points contained in different marking features is the same.

4. The method according to claim 1, wherein The method is processed through a pre-trained semantic detection network; The step of fusing multiple marker features belonging to the same cluster to determine the marker features corresponding to the cluster includes: Determine the weight corresponding to each marker feature; the weight is a parameter adjusted during the training process of the semantic detection network; Based on the feature values of multiple marker features belonging to the same cluster and the weights corresponding to the marker features, the feature value of the fused marker feature corresponding to the cluster is determined.

5. The method according to claim 1, wherein When the number of the target fusion feature maps is M, determining the semantic feature map corresponding to the image to be detected based on the initial semantic feature map and the M target fusion feature maps includes: Upsampling each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine a second feature map corresponding to the N+1th target fusion feature map, wherein when N+1=M, the first feature map corresponding to the Nth target fusion feature map is the Nth target fusion feature map, and N and M are positive integers; The second feature map corresponding to the N+1th target fusion feature map is fused and semantic feature extracted with the Nth target fusion feature map after full connection processing to obtain a first feature map corresponding to the Nth target fusion feature map; wherein, when N=1, the Nth target fusion feature map is the initial semantic feature map; The semantic feature map corresponding to the image to be detected is determined based on the obtained first feature map corresponding to the initial semantic feature map.

6. The method according to claim 5, characterized in that The upsampling of each marker feature in the first feature map corresponding to the N+1th target fusion feature map to determine the second feature map corresponding to the N+1th target fusion feature map includes: According to the clustering result in the process of performing fusion processing to obtain the N+1th target fusion feature map, each label feature in the first feature map corresponding to the N+1th target fusion feature map is up-sampled to determine the second feature map corresponding to the N+1th target fusion feature map.

7. The method according to any one of claims 1 to 6, characterized in that: After determining the semantic feature map corresponding to the image to be detected, the method further includes: Based on the semantic feature map corresponding to the image to be detected, position information of the key points of the image to be detected is determined.

8. An image detection device, characterized in that: include: An acquisition module is used to acquire an image to be detected and determine an initial semantic feature map of the image to be detected; wherein the initial semantic feature map includes multiple marking features; A fusion module is used to perform at least one fusion process on the multiple marking features in the initial semantic feature map, and obtain a target fusion feature map corresponding to the initial semantic feature map after each fusion process; A determination module, configured to determine a semantic feature map corresponding to the image to be detected based on the initial semantic feature map and at least one target fusion feature map, wherein the semantic feature map is used to determine the semantic information of each pixel of the image to be detected; The one-time fusion process includes: Based on the feature similarity between multiple marker features in the target fusion feature map of the previous fusion process, clustering the multiple marker features in the target fusion feature map, fusing the multiple marker features belonging to the same cluster, and determining the marker feature corresponding to the cluster; Perform feature fusion on the marker features corresponding to each cluster and the target fusion feature map to determine an intermediate fusion feature map; Performing deep semantic feature extraction on the intermediate fusion feature map to obtain a target fusion feature map for the current fusion processing; Among them, the target fusion feature map in the first fusion process is the initial semantic feature map.

9. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image detection method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the image detection method according to any one of claims 1 to 7.

Images