Image detection method, related equipment, storage medium and computer product

Abstract

Embodiments of the present application provide an image detection method, related equipment, storage medium, and computer product, the method comprising: acquiring an image to be detected; selecting multiple candidate detection areas from the image to be detected based on pixel features of the image to be detected, selecting an associated detection area that satisfies a target area relationship with a target candidate detection area from the multiple candidate detection areas, and determining a relationship score between the target candidate detection area and the associated detection area based on regional features of the target candidate detection area and regional features of the associated detection area; determining a regional fusion parameter using the feature score and the relationship score of the associated detection area; using the regional fusion parameter, regionally fusing the associated detection area and the target candidate detection area to obtain a fused detection area corresponding to the target candidate detection area, and determining a target detection area for the target object from the fused detection areas corresponding to each candidate detection area, thereby effectively improving the accuracy of target object detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image detection method, related equipment, storage medium and computer product. Background Art

[0002] With the development of computer technology, target detection has become one of the basic tasks in the field of computer vision. Target detection can identify the target of interest and its location in a given image, and is a prerequisite for subsequent higher-level computer vision analysis tasks. Currently, weakly supervised target detection models are mainly used for target detection. First, a series of candidate detection frames that may contain potential targets are extracted. Then, the target detection model is used to filter out the candidate detection frames that are most likely to contain the target as the model detection result. This results in the target detection model also filtering out the detection frames that are most likely to contain the target from a series of detection frames that may contain the target. To a certain extent, this results in only detecting detection frames that contain part of the target from the image. Summary of the Invention

[0003] The embodiments of the present application provide an image detection method, related equipment, storage medium and computer product, which can effectively improve the accuracy of target object detection.

[0004] In one aspect, an embodiment of the present application provides an image detection method, the method comprising:

[0005] Acquire an image to be detected, where the image to be detected includes a target object;

[0006] Selecting a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object;

[0007] Selecting an associated detection area that satisfies a target area relationship with the target candidate detection area from the multiple candidate detection areas, and determining a relationship score between the target candidate detection area and the associated detection area based on the area characteristics of the target candidate detection area and the area characteristics of the associated detection area;

[0008] Obtaining a feature score of the associated detection area, and determining a regional fusion parameter using the feature score of the associated detection area and the relationship score;

[0009] The associated detection area and the target candidate detection area are regionally fused using the regional fusion parameters to obtain a fused detection area corresponding to the target candidate detection area, and a target detection area for the target object is determined from the fused detection areas corresponding to the candidate detection areas.

[0010] In one aspect, an embodiment of the present application provides an image detection device, comprising:

[0011] An acquisition unit, configured to acquire an image to be detected, wherein the image to be detected includes a target object;

[0012] a processing unit, configured to select a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object;

[0013] The processing unit is further configured to select, from the plurality of candidate detection areas, an associated detection area that satisfies a target area relationship with the target candidate detection area, and determine a relationship score between the target candidate detection area and the associated detection area based on regional features of the target candidate detection area and regional features of the associated detection area;

[0014] The processing unit is further configured to obtain a feature score of the associated detection area, and determine a regional fusion parameter using the feature score of the associated detection area and the relationship score;

[0015] The processing unit is further configured to perform regional fusion on the associated detection area and the target candidate detection area using the regional fusion parameter to obtain a fused detection area corresponding to the target candidate detection area, and determine a target detection area for the target object from the fused detection areas corresponding to the candidate detection areas.

[0016] On the one hand, an embodiment of the application provides a computer device, which includes an input device, an output device, a processor and a computer storage medium, wherein the processor and the computer storage medium are interconnected, wherein the computer storage medium is used to store a computer program, and the processor is configured to call the computer program to execute the above-mentioned image detection method.

[0017] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it is used to implement the above-mentioned image detection method.

[0018] On the one hand, an embodiment of the present application provides a computer program product or computer program, the computer program product includes a computer program; the computer program is stored in a computer-readable storage medium, and when the computer program is executed by a processor of a computer device, the above-mentioned image detection method is executed.

[0019] In an embodiment of the present application, a computer device can obtain an image to be detected, and then select multiple candidate detection areas from the image to be detected of the target object based on pixel features of the image to be detected of the target object, select an associated detection area that satisfies the target area relationship with the target candidate detection area from the multiple candidate detection areas of the target object, and determine the relationship score between the target candidate detection area of ​​the target object and the associated detection area of ​​the target object based on the regional features of the target candidate detection area of ​​the target object and the regional features of the associated detection area of ​​the target object; obtain the feature score of the associated detection area of ​​the target object, and use the feature score of the associated detection area of ​​the target object and the target object relationship score to determine the regional fusion parameter; use the target object to select the associated detection area of ​​the target object and the target object relationship score. The target object area fusion parameters are set, and the target object associated detection area and the target object target candidate detection area are regionally fused to obtain the fused detection area corresponding to the target object target candidate detection area, and the target detection area for the target object is determined from the fused detection areas corresponding to the candidate detection areas. By obtaining the relationship scores between different candidate detection areas and regionally fusing the candidate detection areas with the associated detection areas according to the relationship scores between different candidate detection areas and the feature scores of the associated detection areas, the situation where only part of the target object area is detected can be alleviated, thereby effectively detecting the target detection frame including the complete target object and obtaining a target detection area that more accurately includes the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flow chart of an image detection method provided in an embodiment of the present application;

[0022] Figure 2a is a schematic diagram of an image to be detected including multiple candidate detection areas provided by an embodiment of the present application;

[0023] Figure 2b This is a structural diagram of a relationship modeling module provided in an embodiment of the present application;

[0024] Figure 3 This is a flow chart of an image detection method provided in an embodiment of the present application;

[0025] Figure 4a This is a schematic diagram of multiple relationship modeling modules provided in parallel in an embodiment of the present application;

[0026] Figure 4bThis is a schematic diagram of multiple relationship modeling modules connected in series according to an embodiment of the present application;

[0027] Figure 5 This is an architecture diagram of the image detection model provided in the embodiment of the present application;

[0028] Figure 6 Schematic diagram of the detection result of the image to be detected provided by the embodiment of the present application;

[0029] Figure 7 1 is a schematic structural diagram of an image detection device provided in an embodiment of the present application;

[0030] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] An embodiment of the present application provides an image detection solution that can output a target detection area containing a target object in a given image to be detected. The general principle of this image detection solution is as follows: When performing target object detection on any image, multiple candidate detection areas can be first selected from the image; any candidate detection area can include some features of the target object or exclude features of the target object; then, for any candidate detection area, associated detection areas within a certain area can be obtained from the multiple candidate detection areas; then, the regional features of the associated detection areas obtained are used to perform auxiliary discrimination on the candidate detection area, ultimately obtaining a target detection area containing the target object. Specifically, the similarity between any candidate detection area and the associated detection area can be calculated based on the regional features of the candidate detection area and the regional features of the associated detection area, and a relationship score between the candidate detection area and the associated detection area can be determined based on the similarity; then, the target detection area containing the target object can be determined based on the feature score of the associated detection area and the relationship score between the candidate detection area and the associated detection area. The relationship score is used to measure the similarity between any candidate detection area and the associated detection area. The feature score of the associated detection area refers to the score of whether the associated detection area contains the target object.

[0033] Among them, the specific implementation process of determining the target detection area containing the target object based on the feature score of the associated detection area and the relationship score between any candidate detection area and the associated detection area can be: coordinate voting is performed on any candidate detection area based on the relationship score between any candidate detection area and the associated detection area and the feature score of the associated detection area. Specifically, the regional fusion parameter between any candidate detection area and the associated detection area can be obtained based on the relationship score and the feature score of the associated detection area, and then the associated detection area and the target candidate detection area are regionally fused using the regional fusion parameter to obtain the fused detection area corresponding to the target candidate detection area, and the target detection area for the target object is determined from the fused detection areas corresponding to each candidate detection area using the non-maximum suppression method.

[0034] The above image detection scheme has the following beneficial effects: By establishing relationships between different candidate detection areas, it can alleviate the situation where only a portion of the target object is detected, effectively detecting a target detection area that includes the entire target object. Furthermore, based on the relationship score between any candidate detection area and the associated detection area and the feature score of the associated detection area, coordinate voting can be performed on any candidate detection area, resulting in a more accurate target detection area that includes the target object.

[0035] Based on the image detection solution provided above, this application provides an image detection method, please refer to Figure 1 , Figure 1 A flow chart of an image detection method provided in an embodiment of the present application, the image detection method can be executed by a computer device, which can be a terminal device or a server. The terminal device can be a smart phone, tablet computer, laptop computer, desktop computer, smart car, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The image detection method described in this embodiment may include the following steps S101-S105:

[0036] S101: Acquire an image to be detected, where the image to be detected includes a target object.

[0037] The image to be detected can be any static image including one or more objects, any frame image including one or more objects in a dynamic image, or any frame image including one or more objects in a video. Each object in the image to be detected can be referred to as a target object, that is, the number of target objects in the image to be detected can be one or more. For ease of explanation, the following description will be based on an example in which the image to be detected includes one target object. The target object mentioned here can be a person, an animal, etc., and this embodiment of the application does not limit this.

[0038] In a specific implementation, the computer device can acquire images in real time through its configured camera components (such as a camera carried by the computer device, an external camera device, etc.) to obtain the image to be detected; or it can also obtain the image to be detected from the image resources or video resources pre-saved in the local space. Among them, the specific implementation method of obtaining the image to be detected from the image resource can be: if the image resource is a static image containing an object, then the image resource can be directly used as the image to be detected; if the image resource is a dynamic image containing an object, then each frame image in the dynamic image can be subjected to object recognition, thereby selecting a frame image containing the object from the image resource according to the recognition result as the image to be detected. It should be understood that the specific implementation method of obtaining the image to be detected from the video resource is similar to the specific implementation method of obtaining the image to be detected from the dynamic image, and will not be repeated here.

[0039] S102 : selecting a plurality of candidate detection areas from the image to be detected according to pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object.

[0040] The pixel features of the image to be detected refer to the features of the pixel points, which may include but are not limited to features such as color, brightness, and texture. Any candidate detection area may be a candidate detection frame or a closed irregular area. When a candidate detection area does not include the features of the target object, the candidate detection area may be understood as a background frame. For example, Figure 2a As shown, in Figure 2a In the example, multiple candidate detection regions are obtained from the image to be detected. For the target object "phone" in the image to be detected, only the three candidate detection regions in region X1 contain the features of "phone". The rest, such as regions X2 and X3, do not contain the features of "phone".

[0041] In a specific implementation, a computer device can use a selective search model to divide the image to be detected into multiple pixel regions (also known as superpixels) based on the pixel features of the image to be detected. Specifically, the computer device can calculate the similarity between adjacent pixel features of the image to be detected and cluster the pixel points corresponding to the pixel features that meet the similarity conditions to obtain multiple pixel regions. The pixel region is a small region consisting of a series of pixels with adjacent positions and similar characteristics such as color, brightness, and texture. The similarity of the adjacent pixel regions is calculated to obtain the similarity between the adjacent pixel regions. The computer device combines the multiple pixel regions in a bottom-up manner based on the similarity between the pixel regions. In the process of combining the multiple pixel regions, the merged region formed by the intermediate combination is output as a candidate detection region. The output multiple candidate detection regions are used as multiple candidate detection regions that may contain the target object. Specifically, the computer device can merge the adjacent pixel regions that meet the region merging rule to obtain a merged region, wherein the region merging rule includes a rule with the largest similarity, and one merged region is used to determine one candidate detection region. The merged area can be directly determined as a candidate detection area, or the merged area can be processed to obtain a candidate detection area (for example, the merged area is processed as a regular candidate detection area). After obtaining the merged area, the merged area can be regarded as a new pixel area, and then the similarity between the two adjacent pixel areas between the new pixel area and the unmerged pixel area is calculated, and the adjacent pixel areas that meet the area merging rule are merged to obtain a new merged area, and then the new merged area is used to determine a candidate detection area, or the new merged area is processed to obtain a candidate detection area. When all pixel areas are merged to obtain a large area, multiple candidate detection areas can be obtained.

[0042] Among them, the above-mentioned selective search model uses a series of traditional image features for similarity assessment, does not require model training, can better select multiple candidate detection areas from the image to be detected, and the provided candidate detection areas need to have the characteristics of high recall rate and can cover enough real target detection areas. The selective search model can be a model that can generate candidate detection areas without annotating information on the image. For example, the selective search model can be a convolutional neural network (CNN), a multiscale combined grouping model (MCG), etc., which is not limited in this application.

[0043] S103 , selecting an associated detection area that satisfies a target area relationship with the target candidate detection area from the multiple candidate detection areas, and determining a relationship score between the target candidate detection area and the associated detection area based on the area characteristics of the target candidate detection area and the area characteristics of the associated detection area.

[0044] The target region relationship can refer to direct adjacent regions, indirect adjacent regions, etc., for example Figure 2a The candidate detection area 201 and the candidate detection area 202 are directly adjacent to each other, and the candidate detection area 201 and the candidate detection area 203 are indirectly adjacent to each other. The target candidate detection area can be any one of the multiple candidate detection areas, and the number of associated detection areas can be one or more.

[0045] In one embodiment, Figure 2a As shown, there are rich contextual connections between different candidate detection areas. Multiple candidate detection areas corresponding to the same target object (such as the target object "phone") may correspond to different areas of the target object, and candidate detection areas corresponding to different target objects may also provide meaningful contextual information to each other, which can better assist in the classification and recognition of the target object. After the computer device obtains multiple candidate detection areas from the image to be detected, as an implementation method, it can select an associated detection area that satisfies the target relationship with the target candidate detection area from the multiple candidate detection areas, and then obtain the contextual information between the target candidate detection area and the associated detection area, thereby establishing an associated relationship between the target candidate detection area and the associated detection area. In a specific implementation, the computer device can select a candidate detection area that is adjacent or indirectly adjacent to the target candidate detection area from the multiple candidate detection areas as the associated detection area, and then establish an associated relationship between the target candidate detection area and the associated detection area.

[0046] As an implementation method, the intersection-over-union (IoU) ratio (IoU) between the target candidate detection area and each of the multiple candidate detection areas can be directly calculated. Then, based on the IoU ratio between the target candidate detection area and each of the multiple candidate detection areas, an associated detection area that satisfies the target region relationship with the target candidate detection area can be selected. The larger the IoU ratio between two candidate detection areas, the closer the two candidate detection areas are to each other.

[0047] As another embodiment, a relationship modeling module may be used to obtain associated detection areas that satisfy the target region relationship with the target candidate detection area from multiple candidate detection areas. Figure 2b As shown, the relationship modeling module includes three fully connected layers. In the specific implementation, the regional features of all candidate detection areas are respectively passed through three fully connected layers (i.e. Figure 2bThe three branches (K, Q, and V) can be transformed into three different implicit spaces, namely the first implicit space, the second implicit space, and the third implicit space; then, for each second detection area Q, the N nearest candidate detection areas K can be selected from the first detection area K included in the first implicit space based on the intersection-over-union (IoU) between each second detection area Q and each first detection area K, where the value of N can be an integer greater than or equal to 1; then the computer device can map the N candidate detection areas K to obtain the candidate detection areas of each candidate detection area K. The candidate detection areas of each candidate detection area K are then determined as associated detection areas that meet the target area relationship with the target candidate detection area. It should be noted that the selection of the N nearest candidate detection areas K for each second detection area Q based on the IoU takes into account that the target object can be better identified by linking candidate detection areas with different features of the same target object, while avoiding the large amount of video memory consumption when a large number of candidate detection areas are simultaneously modeled.

[0048] After the computer device selects an associated detection area that satisfies a target area relationship with the target candidate detection area from multiple candidate detection areas, the computer device determines a relationship score between the target candidate detection area and the associated detection area based on the regional features of the target candidate detection area and the regional features of the associated detection area. One implementation method is as follows: obtaining the regional features of the target candidate detection area and the regional features of the associated detection area, then calculating the similarity between the target candidate detection area and the associated detection area based on the regional features of the target candidate detection area and the regional features of the associated detection area to obtain the similarity between the target candidate detection area and the associated detection area, and then determining the relationship score between the target candidate detection area and the associated detection area based on the similarity and the similarity and relationship score table. The process of obtaining the regional features of the target candidate detection area and the regional features of the associated detection area can be: in step S102, the computer device first extracts depth image features based on pixel features of the image to be detected using a selective search model, and then uses a pooling technology that can extract accurate regional information (Spatial Pyramid Pooling (SPP) or RoIAlign technology) to extract regional features of each candidate detection area in the depth image features to obtain regional features of each candidate detection area.

[0049] In one embodiment, another implementation method for determining the relationship score between a target candidate detection zone and an associated detection zone based on the regional features of the target candidate detection zone and the regional features of the associated detection zone is to call a relationship modeling module to determine the relationship score between the target candidate detection zone and the associated detection zone based on the regional features of the target candidate detection zone and the regional features of the associated detection zone. In a specific implementation, as previously mentioned, each candidate detection zone is mapped to three different implicit spaces. The computer device can calculate the similarity score between the target second detection zone Q and the candidate detection zone K based on the regional features of the target second detection zone Q and the regional features of the candidate detection zone K. The similarity score between the target second detection zone and the candidate detection zone K is then normalized to obtain the relationship score between the target candidate detection zone and the associated detection zone.

[0050] S104: Acquire feature scores of associated detection areas, and determine regional fusion parameters using the feature scores and relationship scores of the associated detection areas.

[0051] As can be seen from the foregoing, the relationship score between the target candidate detection area and the associated detection area has been determined by the relationship modeling module. The computer device can then call the relationship modeling module to perform weighted processing on the target third detection area V based on the relationship score between the target candidate detection area and the associated detection area, obtaining a weighted feature of the target third detection area V, and then outputting the weighted feature as a new feature of the target candidate detection area. The weighted feature is mapped to two different implicit spaces, and the features of the weighted feature in these two different implicit spaces can be used to make category predictions in two branches. The first branch is to process the features of the weighted feature in one implicit space to obtain the probability that the target candidate detection area belongs to the target object; the other branch is to process the weighted feature in the other implicit space to obtain the probability of the target object's contribution to the target candidate detection area. These two probabilities can measure the degree of association between the target candidate detection area and the target object from different perspectives. After obtaining the probability and contribution probability of a candidate target detection area belonging to the target object, the probability and contribution probability of the candidate target detection area belonging to the target object can be multiplied to obtain the probability of the candidate target detection area, and the probability of the candidate target detection area can be used as the feature score of the candidate target detection area. By multiplying these two probabilities, a more comprehensive description of the probability of the candidate target detection area belonging to the target object can be obtained. In practical applications, when there are multiple target objects, the probability distribution of the candidate target detection area belonging to each target object is obtained based on the features of the fourth implicit space. It should be noted that the feature score for any of the multiple candidate detection areas can be calculated in the same manner as the feature score of the candidate target detection area.

[0052] After obtaining the feature scores of the associated detection areas according to the method for determining the feature scores of the target candidate detection areas, the number of the multiple associated detection areas is one or more. When the number of associated detection areas is one, determining the regional fusion parameter using the feature scores and relationship scores of the associated detection areas may be as follows: the computer device may calculate the product of the feature scores and relationship scores of the associated detection areas to obtain the regional fusion parameter. When the number of associated detection areas is multiple, determining the regional fusion parameter using the feature scores and relationship scores of the associated detection areas may be as follows: for any associated detection area, calculating the product of the feature score of any associated detection area and the relationship score between any associated detection area and the target candidate detection area to obtain the product between any associated detection area and the target candidate detection area, summing the products between each associated detection area and the target candidate detection area to obtain a total score, and then calculating the ratio of the product between each associated detection area and the target candidate detection area to the total score to obtain the regional fusion parameter corresponding to each associated detection area.

[0053] S105 , using regional fusion parameters, regionally fuse the associated detection area and the target candidate detection area to obtain a fused detection area corresponding to the target candidate detection area, and determine a target detection area for the target object from the fused detection areas corresponding to the candidate detection areas.

[0054] In one embodiment, a computer device can obtain the position coordinates of the associated detection area and the position coordinates of the target candidate detection area, and then use the regional fusion parameters as the weighting coefficients of the corresponding associated detection area; and use the weighting coefficients of any associated detection area to perform weighted summation processing on the position coordinates of the corresponding associated detection area to obtain the position coordinates of the fused associated detection area; and then perform regional fusion based on the position coordinates of the target candidate detection area and the position coordinates of the fused associated detection area to obtain a fused detection area. It should be noted that for any candidate detection area among multiple candidate detection areas, the fused detection area can be obtained in the same way as the target candidate detection area, which will not be repeated here. The area corresponding to the fused detection area is larger than the target candidate detection area and the associated detection area.

[0055] After obtaining the fused detection area corresponding to each of the multiple candidate detection areas, a non-maximum suppression algorithm may be used to determine a target detection area for the target object from the fused detection areas corresponding to the respective candidate detection areas.

[0056] In an embodiment of the present application, a computer device can obtain an image to be detected, and then select multiple candidate detection areas from the image to be detected of the target object based on pixel features of the image to be detected of the target object, select an associated detection area that satisfies the target area relationship with the target candidate detection area from the multiple candidate detection areas of the target object, and determine the relationship score between the target candidate detection area of ​​the target object and the associated detection area of ​​the target object based on the regional features of the target candidate detection area of ​​the target object and the regional features of the associated detection area of ​​the target object; obtain the feature score of the associated detection area of ​​the target object, and use the feature score of the associated detection area of ​​the target object and the target object relationship score to determine the regional fusion parameter; use the target object to select the associated detection area of ​​the target object and the target object relationship score. The target object area fusion parameters are set, and the target object associated detection area and the target object target candidate detection area are regionally fused to obtain the fused detection area corresponding to the target object target candidate detection area, and the target detection area for the target object is determined from the fused detection areas corresponding to the candidate detection areas. By obtaining the relationship scores between different candidate detection areas and regionally fusing the candidate detection areas with the associated detection areas according to the relationship scores between different candidate detection areas and the feature scores of the associated detection areas, the situation where only part of the target object area is detected can be alleviated, thereby effectively detecting the target detection frame including the complete target object and obtaining a target detection area that more accurately includes the target object.

[0057] See also Figure 3 , which is a flow chart of an image detection method provided in an embodiment of the present application. The image detection method can be executed by a computer device, which can be a terminal device or a server; the terminal device can be a smart phone, tablet computer, laptop computer, desktop computer, smart car, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The image detection method described in this embodiment may include the following steps S301-S307:

[0058] S301: Acquire an image to be detected, where the image to be detected includes a target object.

[0059] S302 : Select multiple candidate detection areas from the image to be detected based on pixel features of the image to be detected, where any candidate detection area includes some features of the target object or does not include the features of the target object.

[0060] The specific implementation of steps S301-S302 can be found in the above Figure 1The specific implementation of S101-S102 will not be repeated here.

[0061] S303. Map the regional features in each candidate detection area to three implicit spaces respectively, and obtain a first detection area K of each regional feature in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space.

[0062] In a specific implementation, the computer device can map the regional features in each candidate detection area to three implicit spaces respectively through three connection layers, and obtain the first detection area K of each regional feature in the first implicit space, the second detection area Q in the second implicit space, and the third detection area V in the third implicit space. It can be understood that: the computer device maps the regional features in each candidate detection area to the first implicit space through a fully connected layer, and obtains the first detection area K of each regional feature in the first implicit space; the computer device maps the regional features in each candidate detection area to the second implicit space through another fully connected layer, and obtains the second detection area Q of each regional feature in the second implicit space. The computer device maps the regional features in each candidate detection area to the third implicit space through another fully connected layer, and obtains the third detection area V of each regional feature in the third implicit space. Among them, the regional features of any candidate detection area i are f i , after being mapped to the first implicit space, the regional feature of the first detection area of ​​any candidate detection area i in the first implicit space is W k f i , the regional features of all candidate detection areas after being mapped to the first implicit space can form the key matrix K; similarly, the regional features of any candidate detection area i are f i , after being mapped to the second implicit space, the regional feature of the second detection area of ​​any candidate detection area i in the second implicit space is W Q f i , the regional features of all candidate detection areas after being mapped to the second implicit space can form the query matrix Q; the regional feature of any candidate detection area i is f i , after being mapped to the third implicit space, the regional feature of the third detection area of ​​any candidate detection area i in the third implicit space is W V f i , the regional features of all candidate detection areas after being mapped to the second implicit space can form a value matrix V.

[0063] S304: Select a candidate detection area K from the first detection areas K based on the intersection-and-union ratio between the target second detection area Q and any first detection area K, map the candidate detection areas of the candidate detection area K to obtain candidate detection areas, and use the candidate detection areas of the candidate detection area K as associated detection areas that satisfy the target area relationship with the target candidate detection area.

[0064] The target second detection zone Q is the second detection zone Q of the target candidate detection zone in the second implicit space. When the target candidate detection zone is any one of multiple candidate detection zones, the target second detection zone Q is the second detection zone Q of any candidate detection zone in the second implicit space. The number of candidate detection zones K can be one or more.

[0065] In a specific implementation, as can be seen from step S303, each candidate detection zone has a first detection zone K in the first implicit space. The computer device can calculate the union of the target second detection zone Q and any first detection zone K, as well as the intersection between the target second detection zone Q and any first detection zone K. Then, the computer device can calculate the ratio of the intersection to the union between the target second detection zone Q and any first detection zone K to obtain an intersection-and-union ratio between the target second detection zone Q and any first detection zone K. The computer device can then sort the multiple first detection zones K from largest to smallest based on the intersection-and-union ratio between the target second detection zone Q and any first detection zone K, and obtain the top N candidate detection zones K from the sorted results. The value of N is an integer greater than or equal to 1. After selecting a candidate detection zone K from the first detection zones K, the computer device can map the candidate detection zone K to obtain a candidate detection zone of the candidate detection zone K, and determine the candidate detection zone of the candidate detection zone K as an associated detection zone that satisfies the target region relationship with the target candidate detection zone.

[0066] S305. Calculate a similarity score between the target second detection area Q and the candidate detection area K based on the regional features of the target second detection area Q and the regional features of the candidate detection area K. Normalize the similarity score to obtain a relationship score between the target candidate detection area and the associated detection area.

[0067] In a specific implementation, the computer device calculates the cosine similarity between the target second detection area Q and the candidate detection area K based on the regional features of the target second detection area Q and the regional features of the candidate detection area K, normalizes the cosine similarity using a softmax function, and uses the normalized cosine similarity as the similarity score between the target candidate detection area and the associated detection area. The relationship score between the target candidate detection area and the associated detection area is:

[0068]

[0069] Among them, d K is the feature dimension of the target candidate detection area, and softmax() represents normalization.

[0070] S306: Obtain feature scores of the associated detection areas, and determine regional fusion parameters using the feature scores and relationship scores of the associated detection areas.

[0071] In one embodiment, the feature score of a candidate target detection area is determined as follows: the target third detection area V in the third implicit space is weighted based on the relationship score between the candidate target detection area and the associated detection area to obtain weighted features of the target third detection area V. The weighted features of the target third detection area V are then mapped to two implicit spaces, respectively, to obtain features of the weighted features in the fourth implicit space and features in the fifth implicit space. The features of the weighted features in the fourth implicit space and the features in the fifth implicit space can be used to make category predictions in two branches. The first branch is to obtain the probability that the candidate target detection area belongs to the target object based on the features of the fourth implicit space. Specifically, the features of the fourth implicit space can be processed through the activation function softmax to obtain the probability that the candidate target detection area belongs to the target object. The other branch is to obtain the probability of the target object's contribution to the candidate target detection area based on the features of the fifth implicit space. Specifically, the features of the fifth implicit space can be processed through the activation function softmax to obtain the probability that the target object contributes to the candidate target detection area. After obtaining the probability and contribution probability of a target candidate detection zone belonging to the target object, the probability and contribution probability of the target candidate detection zone belonging to the target object can be multiplied to obtain the probability of the target candidate detection zone, and the probability of the target candidate detection zone can be used as the feature score of the target candidate detection zone. By multiplying these two probabilities, the probability of the target candidate detection zone belonging to the target object can be more comprehensively characterized. It should be noted that the feature score for any of the multiple candidate detection zones can be calculated using the same method for determining the feature score of the target candidate detection zone. The method for determining the feature score of the associated detection zone can be similar to the method for determining the feature score of the target candidate detection zone. In this case, the associated detection zone can be used as the target candidate detection zone.

[0072] In one embodiment, when the number of associated detection areas is N, the computer device may first obtain a reference detection area from the N associated detection areas. In a specific implementation, the computer device may calculate the intersection-and-union (IoU) between the target candidate detection area and each associated detection area, then determine whether the IoU between the target candidate detection area and each associated detection area meets a preset condition, and then use the associated detection area corresponding to the IoU that meets the threshold as the reference detection area. As an implementation method, the preset condition is that the IoU is greater than a threshold; the computer device may determine whether the IoU between the target candidate detection area and each associated detection area is greater than the threshold, and use the associated detection area corresponding to the IoU greater than the threshold as the reference detection area. As another implementation method, the preset condition is to take M candidate detection areas from large to small; the computer device sorts the N associated detection areas from large to small according to the intersection-and-union comparison between the target candidate detection area and each associated detection area, obtains the sorting result, and obtains the top M associated detection areas from the sorting result as M reference detection areas; as another implementation method, the preset condition is that the intersection-and-union ratio is greater than a threshold and K are taken from large to small; the computer can use the associated detection areas corresponding to the intersection-and-union ratio greater than the threshold as the initial detection areas, and then sorts the initial detection areas from large to small according to the intersection-and-union comparison between the initial detection areas and the target candidate detection areas to obtain the sorting result, and obtains the top K initial detection areas from the sorting result, and all K initial detection areas are used as reference detection areas.

[0073] After obtaining the reference detection area, the computer device can determine the regional fusion parameters of the reference detection area based on the relationship score between the target candidate detection area and the reference detection area and the feature score of the reference detection area. When the number of reference detection areas is Ni, the regional fusion parameters can be calculated as follows:

[0074]

[0075] Where i represents the target candidate detection area, j represents any reference detection area (1 <j<Ni),w ij represents the regional fusion parameter between the target candidate detection area and any reference detection area j, a ij represents the relationship score between the target candidate detection area and any reference detection area j, s j Represents the feature score of any reference detection area j.

[0076] S307 , using regional fusion parameters, regionally fuse the associated detection area and the target candidate detection area to obtain a fused detection area corresponding to the target candidate detection area, and determine a target detection area for the target object from the fused detection areas corresponding to the candidate detection areas.

[0077] In step S306, a reference detection area is selected from a plurality of associated detection areas. The specific implementation method of step S307 may be: the computer device may use the regional fusion parameters of the reference detection area to regionally fuse the reference detection area and the target candidate detection area to obtain a fused detection area corresponding to the target candidate detection area. Specifically, when there are multiple reference detection areas, the regional fusion feature area of ​​each reference detection area may be determined as the weighting coefficient of the corresponding reference detection area. The weighting coefficient of any reference detection area is then used to perform weighted processing on the position coordinates of the corresponding reference detection area to obtain the position coordinates of any reference detection area after fusion, and regional fusion is performed based on the position coordinates of the target candidate detection area and the position coordinates of any reference detection area after fusion to obtain a fused detection area. Among them, the position coordinates of the fused detection area can be expressed as:

[0078]

[0079] Among them, t represents the position coordinate of the fusion detection area, i represents the target candidate detection area, j represents any reference detection area, and w ij Represents the weighting coefficient of any reference detection area. j Represents the position coordinates of any reference detection area, t i Represents the position coordinates of the target candidate detection area. Where t = {x, y, w, h}, x, y respectively represent the position of the fused detection area point from the coordinate origin; w, h respectively represent the width and height of the fused detection area.

[0080] It should be noted that the above steps S303-S305 can all be executed in the relationship modeling module. The introduction of the relationship modeling module can alleviate the situation where the detection result only contains a part of the target area, and the module is plug-and-play. In the embodiment of the present application, the number of the relationship modeling modules can be one or more. When the number of relationship modeling modules is multiple, such as Figure 4a As shown, multiple relationship modeling modules are connected in parallel to achieve modeling of the relationship between the target candidate detection area and the associated detection area from multiple different angles. Figure 4aIn this embodiment, multiple relationship modeling modules are connected in parallel to a splicing module, which can be used to splice together the reference relationship scores obtained by each relationship modeling module. Each relationship modeling module corresponds to a weighting coefficient, which can be viewed as a learnable vector. In this case, determining the relationship score between the target candidate detection area and the associated detection area may include: calling any one of the multiple relationship modeling modules to determine a reference relationship score between the target candidate detection area and the associated detection area based on the regional characteristics of the target candidate detection area and the regional characteristics of the associated detection area; calling the splicing module to perform a weighted sum of the reference relationship scores determined by each relationship modeling module, and using the score obtained by the weighted sum as the relationship score between the target candidate detection area and the associated detection area. At the same time, the weighted features used in determining the feature scores of the target candidate detection areas are: the reference relationship scores determined by each relationship modeling module are used to weight the regional features of the target third detection area V in the corresponding relationship modeling module to obtain the reference weighted features of the target third detection area V in each relationship modeling module; and the splicing module is called to perform weighted summation on the reference weighted features of the target third detection area V determined by each relationship modeling module to obtain the weighted features of the target third detection area V.

[0081] Alternatively, when the number of relational modeling modules is multiple, such as Figure 4b As shown, multiple relationship modeling modules are connected in series. Each relationship modeling module is connected in series via a fully connected layer. By connecting the relationship modeling modules in series, deeper relationships between the target candidate detection area and the associated detection area can be continuously modeled, modeling the association relationship from a wider range of contextual information. In this case, determining the relationship score between the target candidate detection area and the associated detection area can include: sequentially calling the corresponding relationship modeling modules according to the series connection relationship between the multiple relationship modeling modules to determine a reference relationship score between the target candidate detection area and the associated detection area; and using the reference relationship score output by the last relationship modeling module as the relationship score between the target candidate detection area and the associated detection area.

[0082] Specifically, the computer device calls the first relationship modeling module according to the serial connection relationship between the multiple relationship modeling modules to determine a reference relationship score between the target candidate detection area and the associated detection area based on the regional features of the target candidate detection area and the regional features of the associated detection area. The computer device then uses the reference relationship score to weight the regional features of the target third detection area V to obtain a reference weighted feature of the target third detection area V. The target third detection area V is the third detection area V of the target candidate detection area in the third implicit space. Similarly, the computer device can obtain a reference weighted feature of each third detection area V. The reference weighted feature of each third detection area V is then used as the input of a fully connected layer, and the fully connected layer is called to process the reference weighted feature of each third detection area V to obtain an implicit feature of the reference weighted feature of each third detection area V in the fully connected layer. Then, according to the serial connection relationship between the multiple relationship modeling modules, a second relationship modeling module is called to process the implicit features of each third detection zone V in the fully connected layer based on the reference weighted features. This can be understood as follows: the implicit features of each third detection zone V in the fully connected layer can be understood as the new regional features of each candidate detection zone, which are input into the second relationship modeling module. Then, by calling the first relationship modeling module, steps S303-S305 are executed to obtain the reference relationship score output by the second relationship modeling module and the new weighted features of each third detection zone V. Similarly, the reference relationship score output by the last relationship modeling module is used as the relationship score between the target candidate detection zone and the associated detection zone. Simultaneously, each reference feature output by the last relationship modeling module is weighted and used as the weighted feature of the corresponding third detection zone V when determining the feature score of each candidate detection zone. For example, when determining the feature score of the target candidate detection zone, the reference weighted feature of the target third detection zone V output by the last relationship modeling module is used as the weighted feature for determining the feature score of the target candidate detection zone.

[0083] In an embodiment of the present application, a computer device obtains an associated detection area that satisfies a target region relationship with a target candidate detection area from multiple candidate detection areas. Each candidate detection area is then mapped to three different implicit spaces, resulting in a first detection area K with regional features in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space. Based on the intersection-and-union ratio between the target second detection area Q and any first detection area K, a candidate detection area K is selected from the first detection areas K, and the candidate detection area obtained by mapping the candidate detection area K is an associated detection area that satisfies a target region relationship with the target candidate detection area. Based on the regional features of the target second detection area Q and the regional features of the candidate detection area K, a similarity score is calculated between the target second detection area Q and the candidate detection area K. The similarity score is normalized to obtain a relationship score between the target candidate detection area and the associated detection area. The feature scores of the associated detection areas are obtained, and the feature scores and relationship scores of the associated detection areas are used to determine regional fusion parameters. Using the regional fusion parameters, the associated detection areas and the target candidate detection areas are regionally fused to obtain the fused detection areas corresponding to the target candidate detection areas. The target detection area for the target object is then determined from the fused detection areas corresponding to the candidate detection areas. Relationship features can be extracted from different candidate detection areas using three implicit spaces, alleviating the situation where only a portion of the target object is detected, effectively detecting the entire target object. Furthermore, different candidate detection areas can be merged to obtain a more accurate target detection area that includes the target object.

[0084] Based on the image detection method provided above, this application provides an image detection model, which can be Figure 5 As shown, the image detection model may include: a convolutional neural network CNN (or MCG), a relationship modeling module, and multiple fully connected layers (FC). The image detection model can be used to detect the target object in the image to be detected. It should be noted that the architecture of the image detection model provided in the embodiment of the present application is not limited to Figure 5The model architecture shown in the figure can be a model capable of extracting high-resolution feature maps with strong semantic information. The general principle of implementing the image detection method using this image detection model is as follows: A computer device invokes a convolutional neural network to extract deep image features from the image to be detected. These deep image features are features specific to the entire image. The deep image features are then extracted using the SPP pooling technique to extract regional features of candidate detection areas, resulting in multiple candidate detection areas and regional features for each candidate detection area. The regional features of each candidate detection area are then processed through two fully connected layers, and then the processed regional features of each candidate detection area are processed through a relationship modeling module. Specifically, the regional features of each processed candidate detection area are mapped into three implicit spaces, resulting in a first detection area K in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space. A candidate detection area K is then selected from the first detection areas K based on the intersection-over-union ratio between the target second detection area Q and any first detection area K. The candidate detection areas of the mapped candidate detection areas K are associated detection areas that satisfy a target region relationship with the target candidate detection area. Based on the regional features of the target second detection area Q and the regional features of the candidate detection area K, the similarity score between the target second detection area Q and the candidate detection area K is calculated; the similarity score is normalized, and the normalized similarity score is taken out from the relationship modeling module as the relationship score between the target candidate detection area and the associated detection area.

[0085] Then, based on the relationship score between the candidate target detection region and the associated detection region, the third target detection region V can be weighted to obtain its weighted features. These weighted features are then mapped to two implicit spaces through two fully connected layers. The regional features of the third target detection region V in these two implicit spaces can be used to make category predictions in two branches. Branch one: The computer device performs softmax processing on the features of the first implicit space to predict the probability that the target candidate detection belongs to the target object. Branch two: The computer device performs softmax processing on the features of the second implicit space to obtain the probability that the target object contributes to the candidate target detection region. These two probabilities measure the degree of association between the candidate target detection region and the target object from different perspectives. By multiplying these two probabilities, a more comprehensive probability of the candidate target detection region belonging to the target object can be obtained. The probability of the candidate target detection region belonging to the target object is then used as the feature score of the candidate target detection region. Similarly, the feature score of each candidate detection region can be obtained. After obtaining the feature score for each detection area and the relationship score between each candidate detection area and its corresponding associated detection area, the fused detection area corresponding to each candidate detection area is calculated based on the feature score and the relationship score between each candidate detection area and its corresponding associated detection area. The target detection area containing the target object is then determined from the fused detection areas corresponding to each candidate detection area using a non-maximum suppression algorithm. Furthermore, this image detection model can output not only the target detection area containing the target object in the image to be detected, but also the category of the target object.

[0086] In one embodiment, the image detection model is trained based on sample images. In the embodiment of the present application, the image detection model does not need to provide a target detection area including sample objects during the model training phase. It only needs to provide all sample object categories contained in each sample image. This is of great significance in implementing image detection model training under large-scale training images, because it avoids a large amount of manual labeling, and also avoids the need for industry experts to assist in labeling some long-tail categories, which is time-consuming and labor-intensive, and difficult to meet the large amount of training data required by deep learning models. The training process of the entire image detection model is as follows:

[0087] (1) Obtain a sample image, which includes multiple sample object labels.

[0088] (2) Using the convolutional neural network in the initial image detection model to perform unsupervised learning on the sample image, a plurality of sample detection areas are generated, each of which includes some features of any sample object or does not include any features of any sample object. The architecture diagram of the initial image detection model can be found in Figure 5 The architecture diagram of the image detection model shown.

[0089] (3) For the existing multiple sample detection areas, a multi-instance learning method is used to utilize image-level labels for supervised learning. Specifically, the relationship modeling module in the initial image detection model is first used to process the multiple sample detection areas to obtain the relationship score between each sample detection area and the corresponding sample detection area. Then, the weighted features of the third detection area V of each sample detection area in the third implicit space in the relationship modeling module are obtained. The weighted features of the third detection area V of each sample detection area in the third implicit space are respectively mapped to two implicit spaces through two fully connected layers to obtain the features of each weighted feature in the fourth implicit space and the features of each weighted feature in the fifth implicit space. Then, the features of each weighted feature in the fourth implicit space are subjected to softmax processing to obtain the probability distribution of each detection area belonging to each sample object. The features of each weighted feature in the fifth implicit space are subjected to softmax processing to obtain the probability of each sample object belonging to any sample detection area. Then, the probability distribution of each detection area belonging to each sample object and the probability of each sample object belonging to any sample detection area are multiplied to obtain the probability of each sample detection area belonging to any sample object. The probability of each sample detection area belonging to any sample object is averaged to obtain the probability distribution of the sample image. The probability distribution of the sample image can be understood as the probability distribution of the combination of the probabilities of each category.

[0090] (3) The computer device regards the multiple sample detection areas as a package and labels the package with multiple target object labels of the sample image; then uses a multi-instance learning method to supervise the initial image detection model based on the probability distribution of the sample image and the multiple target object labels to obtain an image detection model.

[0091] There are two main approaches to training image detection models. For image detection models built on deep convolutional neural networks, training a robust, high-accuracy model requires a large amount of labeled data. Each object of interest in an image must be labeled with a bounding box and its category, which requires significant annotation effort and manpower. Some rare categories even require the intervention of domain experts. To alleviate the large amount of labeled data required for training general-purpose image detection models, weakly supervised object detection methods have gradually attracted the attention of researchers. Some methods require only the object categories contained in an image, while others incorporate human interaction and feedback to train image detection models. These methods eliminate the need for precise object location information, significantly reducing the complexity of labeling. This allows us to effectively leverage the vast amount of images available online for model training, enabling detection of large-scale / long-tail categories. Similar to fully supervised methods, weakly supervised object detection methods also filter the most likely detection area from a series of possible object boxes. Consequently, they can often fail to detect the entire object, limiting the accuracy of image detection models. The image detection model training provided by the embodiment of the present application is a weakly supervised target detection method based on modeling the relationship between instances, which establishes the relationship between different sample detection areas and can enhance the regional features of a single sample candidate frame. This is conducive to using the enhanced regional features of a single sample candidate frame to perform multi-instance learning on the initial image detection model. The resulting image detection model is more accurate. During the model training phase, it is not necessary to provide the actual target detection area. Instead, it is only necessary to provide all the sample object categories contained in each image. This is of great significance in implementing image detection model training under large-scale training images because it avoids a large amount of manual labeling, especially for some long-tail categories. The labeling of these categories often requires the assistance of industry experts, which is time-consuming and labor-intensive, and it is difficult to meet the large amount of training data required by deep learning models.

[0092] In actual application, the image detection method and image detection model provided in the embodiment of the present application can be applied to the recognition of target objects in images, and can also be applied to quality inspection of the U-image industry, that is, according to the image detection method and image detection model, defects can be detected on a single image to obtain the location of the defect in the image (that is, the detection box in the figure) and the defect type, for example Figure 6 As shown, the Figure 6 The image to be detected is detected by the image detection method and the image detection model, and the location of the defect in the image to be detected is obtained, that is, the area framed by the target detection frame in the image to be detected, and the defect type is NG.

[0093] Based on the image detection method provided above, please see Figure 7 , which is a structural diagram of an image detection device provided in an embodiment of the present application. Figure 7As shown, the image detection device can be applied to the above Figure 1 or Figure 3 The computer device in the corresponding embodiment; specifically, the image detection device can be a computer program (including program code) running in the computer device, for example, the image detection device is an application software; the image detection device can be used to execute the corresponding steps in the method provided in the embodiment of the present application.

[0094] An acquisition unit 701 is configured to acquire an image to be detected, where the image to be detected includes a target object.

[0095] The processing unit 702 is configured to select a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object;

[0096] The processing unit 702 is further configured to select, from the plurality of candidate detection areas, an associated detection area that satisfies a target area relationship with the target candidate detection area, and determine a relationship score between the target candidate detection area and the associated detection area based on regional features of the target candidate detection area and regional features of the associated detection area;

[0097] The processing unit 702 is further configured to obtain a feature score of the associated detection area, and determine a region fusion parameter using the feature score of the associated detection area and the relationship score;

[0098] The processing unit 702 is further configured to perform regional fusion on the associated detection area and the target candidate detection area using the regional fusion parameter to obtain a fused detection area corresponding to the target candidate detection area, and determine a target detection area for the target object from the fused detection areas corresponding to the candidate detection areas.

[0099] In one embodiment, when selecting an associated detection area that satisfies a target region relationship with the target candidate detection area from the multiple candidate detection areas, the processing unit 702 may be specifically configured to:

[0100] Mapping the regional features in each candidate detection area into three implicit spaces respectively, obtaining a first detection area K of each regional feature in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space;

[0101] Select a candidate detection area K from the first detection area K according to the intersection-and-union ratio between the target second detection area Q and any first detection area K, and map the candidate detection area of ​​the candidate detection area K to obtain the candidate detection area. The target second detection area Q is the second detection area Q of the target candidate detection area in the second implicit space.

[0102] The candidate detection area of ​​the candidate detection area K is used as an associated detection area that satisfies a target area relationship with the target candidate detection area.

[0103] In one embodiment, when determining the relationship score between the candidate target detection area and the associated detection area based on the regional characteristics of the candidate target detection area and the regional characteristics of the associated detection area, the processing unit 702 may be specifically configured to:

[0104] Calculating a similarity score between the target second detection area Q and the candidate detection area K based on the regional features of the target second detection area Q and the regional features of the candidate detection area K;

[0105] The similarity scores are normalized to obtain a relationship score between the candidate target detection area and the associated detection area.

[0106] In one embodiment, the association detection area and the relationship score are both determined by calling a relationship modeling module, and the number of the relationship modeling modules is one or more;

[0107] When there are multiple relationship modeling modules, the multiple relationship modeling modules are connected in parallel, and the multiple relationship modeling modules are connected to the splicing module after being connected in parallel;

[0108] Alternatively, the multiple relationship modeling modules are connected in series.

[0109] In one embodiment, if the multiple relationship modeling modules are connected in parallel, the relationship score is determined by:

[0110] The processing unit 702 is configured to call any one of the multiple relationship modeling modules to determine a reference relationship score between the candidate target detection area and the associated detection area based on the regional characteristics of the candidate target detection area and the regional characteristics of the associated detection area; call the splicing module to perform a weighted sum of the reference relationship scores determined by each relationship modeling module, and use the score obtained by the weighted sum as the relationship score between the candidate target detection area and the associated detection area.

[0111] In one embodiment, if the multiple relationship modeling modules are connected in series, the relationship score is determined by:

[0112] The processing unit 702 is configured to sequentially call corresponding relationship modeling modules according to the serial connection relationship between the multiple relationship modeling modules to determine a reference relationship score between the candidate target detection area and the associated detection area; and use the reference relationship score output by the last relationship modeling module as the relationship score between the candidate target detection area and the associated detection area.

[0113] In one embodiment, there are multiple associated detection areas. When the processing unit 7102 uses the regional fusion parameters to perform regional fusion on the associated detection areas and the target candidate detection areas to obtain a fused detection area corresponding to the target candidate detection area, the processing unit 7102 may be specifically configured to:

[0114] Obtaining the position coordinates of each associated detection area and the position coordinates of the target candidate detection area;

[0115] The regional fusion parameter is used as a weight coefficient of the corresponding associated detection area, and the weight coefficient of any associated detection area is used to perform weighted summation processing on the position coordinates of the corresponding associated detection area to obtain the position coordinates of any associated detection area after fusion;

[0116] Based on the position coordinates of the target candidate detection area and the position coordinates of any associated detection area after fusion, regional fusion is performed to obtain a fused detection area.

[0117] In one embodiment, when the processing unit 702 selects a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, it may be specifically configured to:

[0118] Dividing the image to be detected into a plurality of pixel regions according to pixel features of the image to be detected;

[0119] A similarity calculation is performed on two adjacent pixel regions to obtain the similarity between the two adjacent pixel regions, and adjacent pixel regions that meet the region merging rule are merged to obtain a merged region, wherein the region merging rule includes a rule with the maximum similarity, and a merged region is used to determine a candidate detection area.

[0120] In one embodiment, the feature score is determined by:

[0121] The processing unit 702 is configured to weight the regional features of the target third detection area V using the relationship score to obtain the weighted features of the target third detection area V; the target third detection area V is the third detection area V of the target candidate detection area in the third implicit space;

[0122] Mapping the weighted features to two implicit spaces respectively to obtain features of the weighted features in a fourth implicit space and features in a fifth implicit space;

[0123] Obtaining a probability that the target candidate detection area belongs to the target object based on the characteristics of the fourth implicit space;

[0124] Obtaining a contribution probability of the target object to the target candidate detection area based on features of the fifth implicit space;

[0125] Multiplying the probability that the target candidate detection area belongs to the target object by the contribution probability to obtain the probability of the target candidate detection area;

[0126] The probability of the target candidate detection area is used as the feature score of the target candidate detection area.

[0127] It is understandable that the functions of the functional units of the image detection device of this embodiment can be specifically implemented according to the method in the above method embodiment, and the specific implementation process can refer to the above method embodiment. Figure 1 or Figure 3 The relevant description will not be repeated here.

[0128] Further, see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 1 or Figure 3 The computer device in the corresponding embodiment can be Figure 8 The computer equipment shown. Figure 8 As shown, the computer device may include: a processor 801, an input device 802, an output device 803, and a memory 804. The processor 801, input device 802, output device 803, and memory 804 are connected via a bus 805. The memory 804 is used to store a computer program, which includes program instructions. The processor 801 is used to execute the program instructions stored in the memory 804. Optionally, the computer device may also include a camera.

[0129] In the embodiment of the present application, the processor 801 performs the following operations by running the executable program code in the memory 804:

[0130] Acquire an image to be detected, where the image to be detected includes a target object;

[0131] Selecting a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object;

[0132] Selecting an associated detection area that satisfies a target area relationship with the target candidate detection area from the multiple candidate detection areas, and determining a relationship score between the target candidate detection area and the associated detection area based on the area characteristics of the target candidate detection area and the area characteristics of the associated detection area;

[0133] Obtaining a feature score of the associated detection area, and determining a regional fusion parameter using the feature score of the associated detection area and the relationship score;

[0134] The associated detection area and the target candidate detection area are regionally fused using the regional fusion parameters to obtain a fused detection area corresponding to the target candidate detection area, and a target detection area for the target object is determined from the fused detection areas corresponding to the candidate detection areas.

[0135] In one embodiment, when the processor 801 selects an associated detection area that satisfies a target region relationship with the target candidate detection area from the multiple candidate detection areas, it may be specifically configured to:

[0136] Mapping the regional features in each candidate detection area into three implicit spaces respectively, obtaining a first detection area K of each regional feature in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space;

[0137] Select a candidate detection area K from the first detection area K according to the intersection-and-union ratio between the target second detection area Q and any first detection area K, and map the candidate detection area of ​​the candidate detection area K to obtain the candidate detection area. The target second detection area Q is the second detection area Q of the target candidate detection area in the second implicit space.

[0138] The candidate detection area of ​​the candidate detection area K is used as an associated detection area that satisfies a target area relationship with the target candidate detection area.

[0139] In one embodiment, when determining the relationship score between the candidate target detection area and the associated detection area based on the regional characteristics of the candidate target detection area and the regional characteristics of the associated detection area, the processor 801 may be specifically configured to:

[0140] Calculating a similarity score between the target second detection area Q and the candidate detection area K based on the regional features of the target second detection area Q and the regional features of the candidate detection area K;

[0141] The similarity scores are normalized to obtain a relationship score between the candidate target detection area and the associated detection area.

[0142] In one embodiment, the association detection area and the relationship score are both determined by calling a relationship modeling module, and the number of the relationship modeling modules is one or more;

[0143] When there are multiple relationship modeling modules, the multiple relationship modeling modules are connected in parallel, and the multiple relationship modeling modules are connected to the splicing module after being connected in parallel;

[0144] Alternatively, the multiple relationship modeling modules are connected in series.

[0145] In one embodiment, if the multiple relationship modeling modules are connected in parallel, the relationship score is determined by:

[0146] The processor 801 is configured to call any one of the multiple relationship modeling modules to determine a reference relationship score between the target candidate detection area and the associated detection area based on the regional characteristics of the target candidate detection area and the regional characteristics of the associated detection area; call the splicing module to perform a weighted sum of the reference relationship scores determined by each relationship modeling module, and use the score obtained by the weighted sum as the relationship score between the target candidate detection area and the associated detection area.

[0147] In one embodiment, if the multiple relationship modeling modules are connected in series, the relationship score is determined by:

[0148] The processor 801 is configured to call corresponding relationship modeling modules in sequence according to the serial connection relationship between the multiple relationship modeling modules to determine a reference relationship score between the candidate target detection area and the associated detection area;

[0149] The reference relationship score output by the last relationship modeling module is used as the relationship score between the target candidate detection area and the associated detection area.

[0150] In one embodiment, there are multiple associated detection areas. When the processor 801 uses the regional fusion parameter to perform regional fusion on the associated detection area and the target candidate detection area to obtain a fused detection area corresponding to the target candidate detection area, the processor 801 may be specifically configured to:

[0151] Obtaining the position coordinates of each associated detection area and the position coordinates of the target candidate detection area;

[0152] The regional fusion parameter is used as a weight coefficient of the corresponding associated detection area, and the weight coefficient of any associated detection area is used to perform weighted summation processing on the position coordinates of the corresponding associated detection area to obtain the position coordinates of any associated detection area after fusion;

[0153] Based on the position coordinates of the target candidate detection area and the position coordinates of any associated detection area after fusion, regional fusion is performed to obtain a fused detection area.

[0154] In one embodiment, it is characterized in that, when the processor 801 selects multiple candidate detection areas from the image to be detected based on the pixel features of the image to be detected, it can be specifically used to:

[0155] Dividing the image to be detected into a plurality of pixel regions according to pixel features of the image to be detected;

[0156] A similarity calculation is performed on two adjacent pixel regions to obtain the similarity between the two adjacent pixel regions, and adjacent pixel regions that meet the region merging rule are merged to obtain a merged region, wherein the region merging rule includes a rule with the maximum similarity, and a merged region is used to determine a candidate detection area.

[0157] In one embodiment, the feature score is determined by:

[0158] The processor 801 is configured to weight regional features of a target third detection zone V using the relationship score to obtain weighted features of the target third detection zone V; the target third detection zone V is the third detection zone V of the target candidate detection zone in the third implicit space;

[0159] Mapping the weighted features to two implicit spaces respectively to obtain features of the weighted features in a fourth implicit space and features in a fifth implicit space;

[0160] Obtaining a probability that the target candidate detection area belongs to the target object based on the characteristics of the fourth implicit space;

[0161] Obtaining a contribution probability of the target object to the target candidate detection area based on features of the fifth implicit space;

[0162] Multiplying the probability that the target candidate detection area belongs to the target object by the contribution probability to obtain the probability of the target candidate detection area;

[0163] The probability of the target candidate detection area is used as the feature score of the target candidate detection area.

[0164] It should be understood that in the embodiment of the present application, the processor 801 may be a central processing unit (CPU), and the processor 801 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0165] The memory 804 may include a read-only memory and a random access memory, and provides instructions and data to the processor 801. A portion of the memory 804 may also include a nonvolatile random access memory.

[0166] The input device 802 can be connected to a camera and input an image to be detected to the processor 801 ; the output device 803 can include a display, etc.

[0167] In a specific implementation, the processor 801, input device 802, output device 803 and memory 804 described in the embodiments of the present application can execute the implementation methods described in all the above embodiments, and can also execute the implementation methods described in the above apparatus, which will not be repeated here.

[0168] In an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the steps performed in all the above embodiments can be executed.

[0169] An embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. When the computer instructions are executed by a processor of a computer device, the methods in all the above embodiments are executed.

[0170] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0171] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. An image detection method, characterized in that: include: Acquire an image to be detected, where the image to be detected includes a target object; Selecting a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object; Selecting an associated detection area that satisfies a target area relationship with a target candidate detection area from the multiple candidate detection areas, and determining a relationship score between the target candidate detection area and the associated detection area based on regional characteristics of the target candidate detection area and regional characteristics of the associated detection area; the target candidate detection area is any one of the multiple candidate detection areas; Obtaining a feature score of the associated detection area, and determining a regional fusion parameter using the feature score of the associated detection area and the relationship score; The associated detection area and the target candidate detection area are regionally fused using the regional fusion parameters to obtain a fused detection area corresponding to the target candidate detection area, and a target detection area for the target object is determined from the fused detection areas corresponding to the candidate detection areas.

2. The method according to claim 1, wherein The selecting, from the plurality of candidate detection areas, an associated detection area that satisfies a target area relationship with the target candidate detection area includes: Mapping the regional features in each candidate detection area into three implicit spaces respectively, obtaining a first detection area K of each regional feature in the first implicit space, a second detection area Q in the second implicit space, and a third detection area V in the third implicit space; Select a candidate detection area K from the first detection area K according to the intersection-and-union ratio between the target second detection area Q and any first detection area K, and map the candidate detection area of ​​the candidate detection area K to obtain the candidate detection area. The target second detection area Q is the second detection area Q of the target candidate detection area in the second implicit space. The candidate detection area of ​​the candidate detection area K is used as an associated detection area that satisfies a target area relationship with the target candidate detection area.

3. The method according to claim 2, wherein The determining, based on the regional characteristics of the candidate target detection area and the regional characteristics of the associated detection area, a relationship score between the candidate target detection area and the associated detection area includes: Calculating a similarity score between the target second detection area Q and the candidate detection area K based on the regional features of the target second detection area Q and the regional features of the candidate detection area K; The similarity scores are normalized to obtain a relationship score between the candidate target detection area and the associated detection area.

4. The method according to claim 1, wherein The association detection area and the relationship score are both determined by calling a relationship modeling module, and the number of the relationship modeling modules is one or more; When there are multiple relationship modeling modules, the multiple relationship modeling modules are connected in parallel, and the multiple relationship modeling modules are connected to the splicing module after being connected in parallel; Alternatively, the multiple relationship modeling modules are connected in series.

5. The method according to claim 4, wherein If the multiple relationship modeling modules are connected in parallel, the relationship score is determined in the following manner: calling any one of the plurality of relationship modeling modules to determine a reference relationship score between the candidate target detection area and the associated detection area based on the regional characteristics of the candidate target detection area and the regional characteristics of the associated detection area; The splicing module is called to perform weighted summation on the reference relationship scores determined by each relationship modeling module, and the score obtained by the weighted summation is used as the relationship score between the target candidate detection area and the associated detection area.

6. The method according to claim 4, wherein If the multiple relationship modeling modules are connected in series, the relationship score is determined in the following manner: According to the serial connection relationship between the plurality of relationship modeling modules, calling the corresponding relationship modeling modules in sequence to determine the reference relationship score between the candidate target detection area and the associated detection area; The reference relationship score output by the last relationship modeling module is used as the relationship score between the target candidate detection area and the associated detection area.

7. The method according to claim 1, wherein There are multiple associated detection areas, and using the regional fusion parameters to perform regional fusion on the associated detection areas and the target candidate detection areas to obtain fused detection areas corresponding to the target candidate detection areas includes: Obtaining the position coordinates of each associated detection area and the position coordinates of the target candidate detection area; The regional fusion parameter is used as a weight coefficient of the corresponding associated detection area, and the weight coefficient of any associated detection area is used to perform weighted summation processing on the position coordinates of the corresponding associated detection area to obtain the position coordinates of any associated detection area after fusion; Based on the position coordinates of the target candidate detection area and the position coordinates of any associated detection area after fusion, regional fusion is performed to obtain a fused detection area.

8. The method according to claim 1, wherein The step of selecting a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected comprises: Dividing the image to be detected into a plurality of pixel regions according to pixel features of the image to be detected; A similarity calculation is performed on two adjacent pixel regions to obtain the similarity between the two adjacent pixel regions, and adjacent pixel regions that meet the region merging rule are merged to obtain a merged region, wherein the region merging rule includes a rule with the greatest similarity, and a merged region is used to determine a candidate detection area.

9. The method according to claim 3, wherein The method for determining the feature score includes: The relationship score is used to weight the regional features of the target third detection area V to obtain the weighted features of the target third detection area V; the target third detection area V is the third detection area V of the target candidate detection area in the third implicit space; Mapping the weighted features to two implicit spaces respectively to obtain features of the weighted features in a fourth implicit space and features in a fifth implicit space; Obtaining a probability that the target candidate detection area belongs to the target object based on the characteristics of the fourth implicit space; Obtaining a contribution probability of the target object to the target candidate detection area based on features of the fifth implicit space; Multiplying the probability that the target candidate detection area belongs to the target object by the contribution probability to obtain the probability of the target candidate detection area; The probability of the target candidate detection area is used as the feature score of the target candidate detection area.

10. An image detection device, characterized in that: include: An acquisition unit, configured to acquire an image to be detected, wherein the image to be detected includes a target object; a processing unit, configured to select a plurality of candidate detection areas from the image to be detected based on pixel features of the image to be detected, wherein any candidate detection area includes some features of the target object or does not include the features of the target object; The processing unit is further configured to select, from the plurality of candidate detection areas, an associated detection area that satisfies a target area relationship with the target candidate detection area, and determine a relationship score between the target candidate detection area and the associated detection area based on regional characteristics of the target candidate detection area and regional characteristics of the associated detection area; the target candidate detection area is any one of the plurality of candidate detection areas; The processing unit is further configured to obtain a feature score of the associated detection area, and determine a regional fusion parameter using the feature score of the associated detection area and the relationship score; The processing unit is further configured to perform regional fusion on the associated detection area and the target candidate detection area using the regional fusion parameter to obtain a fused detection area corresponding to the target candidate detection area, and determine a target detection area for the target object from the fused detection areas corresponding to the candidate detection areas.

11. A computer device, characterized in that: include: a processor adapted to implement one or more computer programs; A computer storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by the processor and executing the image detection method according to any one of claims 1 to 9.

12. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by a processor, the computer program is used to load and execute the image detection method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program is used to load and execute the image detection method according to any one of claims 1 to 9.

Images