Image detection method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image detection method and device, electronic equipment and storage medium, relates to the technical field of artificial intelligence, and particularly relates to the fields of automatic driving, intelligent transportation, computer vision and the like. The implementation scheme is as follows: obtaining a target image to be detected, the target image comprising a plurality of objects; performing feature extraction on the target image to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of prior boxes corresponding to a plurality of scales; for each of the plurality of features corresponding to the plurality of feature scales, obtaining a sub-feature corresponding to each of the plurality of prior boxes corresponding to the feature; obtaining a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features; and obtaining a plurality of predicted boxes of the target image based on the plurality of target sub-features, each of the plurality of predicted boxes indicating one of the plurality of objects.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of autonomous driving, intelligent transportation, computer vision, and more specifically to an image detection method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0003] Artificial intelligence-based image detection technology obtains image features through image detection models, and obtains objects contained in the image (for example, people, vehicles, etc.) based on the image features. It has been widely used in various scenarios.

[0004] The approaches described in this section are not necessarily approaches that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any approach described in this section is prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues raised in this section should not be considered as having been recognized in any prior art. Summary of the Invention

[0005] The present disclosure provides an image detection method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] According to one aspect of the present disclosure, an image detection method is provided, comprising: obtaining a target image to be detected, the target image including a plurality of objects; performing feature extraction on the target image to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of prior frames, the plurality of prior frames corresponding to a plurality of scales;

[0007] For each of the multiple features corresponding to the multiple feature scales, obtain a sub-feature corresponding to each of the multiple prior boxes corresponding to the feature; obtain a plurality of target sub-features from the multiple sub-features corresponding to the multiple features; and obtain a plurality of prediction boxes of the target image based on the multiple target sub-features, each of the plurality of prediction boxes indicating one of the multiple objects.

[0008] According to another aspect of the present disclosure, there is provided an image detection apparatus, comprising: a target image obtaining unit configured to obtain a target image to be detected, the target image comprising a plurality of objects; a feature extracting unit configured to perform feature extraction on the target image to be detected to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of prior boxes corresponding to a plurality of scales; a sub-feature obtaining unit configured to, for each of a plurality of features corresponding to the plurality of feature scales, obtain sub-features corresponding to each of a plurality of prior boxes corresponding to the feature; a target sub-feature obtaining unit configured to obtain a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features; and a predicted box obtaining unit configured to obtain a plurality of predicted boxes of the target image based on the plurality of target sub-features, each of the plurality of predicted boxes indicating one of the plurality of objects.

[0009] According to another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method according to an embodiment of the present disclosure.

[0010] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method according to an embodiment of the present disclosure.

[0011] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to an embodiment of the present disclosure.

[0012] According to one or more embodiments of the present disclosure, the range of scales of objects in an image to be detected in image detection can be expanded, so that target detection can be implemented on objects of more scales.

[0013] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in

[0015] Figure 1 A schematic diagram illustrating an exemplary system in which various methods described herein can be implemented according to embodiments of the present disclosure is shown;

[0016] Figure 2 A flowchart illustrating a process of obtaining a plurality of target sub-features from a plurality of sub-features corresponding to a plurality of features in an image processing method according to embodiments of the present disclosure is shown;

[0017] Figure 3 A flowchart illustrating a process of obtaining a plurality of target sub-features from a plurality of sub-features corresponding to a plurality of features in an image processing method according to embodiments of the present disclosure is shown;

[0018] Figure 4 A flowchart illustrating a process of obtaining a plurality of predicted bounding boxes of a target image based on a plurality of target sub-features in an image processing method according to embodiments of the present disclosure is shown;

[0019] Figure 5 A schematic diagram illustrating a process of an image processing method according to embodiments of the present disclosure is shown;

[0020] Figure 6 A flowchart illustrating a process of obtaining a plurality of target sub-features based on a predicted probability corresponding to each of a plurality of sub-features corresponding to each of a plurality of features in an image processing method according to embodiments of the present disclosure is shown;

[0021] Figure 7 A flowchart illustrating a process of obtaining a plurality of predicted bounding boxes of a target image based on a plurality of target sub-features in an image processing method according to embodiments of the present disclosure is shown;

[0022] Figure 8 A block diagram illustrating a structure of an image detection device according to embodiments of the present disclosure is shown;

[0023] Figure 9 A block diagram illustrating an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure are described herein below with reference to the accompanying drawings, in which various details are set forth to facilitate an understanding of the present disclosure. It should be readily apparent to those of ordinary skill in the art that the embodiments described herein are merely exemplary of the present disclosure and a great variety of modifications and alterations to the embodiments described herein can be construed as falling within the scope of the present disclosure. Likewise, the terms "comprise," "comprises," "comprising," "include," "includes," and "including" as may be used in the specification and / or the claims are intended to be open-ended, i.e., to mean including, but not limited to. Therefore, use of such terms in the specification and / or the claims is not intended to be limiting.

[0025] In the present disclosure, the terms "first", "second", etc. are used to describe various elements only for the purpose of distinguishing one element from another, and are not intended to imply a relative importance or a specific spatial, chronological, or

[0026] The terminology used in the description of the various described examples herein is for the purpose of describing particular embodiments only and is not intended to be limiting. Unless specifically defined herein, any terms used herein that are not specifically defined are intended to be given their ordinary meaning to a person of ordinary skill in the art. Moreover, the use of "including", "comprising", "having" "implying" "involving", "characterized by", "characterized into" and variations thereof herein, is meant to encompass the items listed thereafter, and equivalents thereof, as well as additional items. Furthermore, "and / or" where used herein is used to mean either "and" or "or", or both, depending upon the context.

[0027] Embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0028] Figure 1 A schematic diagram of an exemplary system 100 in which various methods and apparatus described herein can be implemented in accordance with embodiments of the present disclosure is shown. Referring to Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 that couple the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more application programs.

[0029] In embodiments of the present disclosure, the server 120 can run one or more services or software applications that enable the execution of image detection methods in accordance with the present disclosure.

[0030] In certain embodiments, the server 120 can also provide other services or software applications, which can include non-virtual and virtual environments. In certain embodiments, these services can be provided as web-based services or cloud services, for example, to users of the client devices 101, 102, 103, 104, 105, and / or 106 under a software-as-a-service (SaaS) model.

[0031] In Figure 1 In the illustrated configuration, the server 120 can include one or more components implementing functionality performed by the server 120. These components can include software components, hardware components, or a combination thereof, executable by one or more processors. Users operating the client devices 101, 102, 103, 104, 105, and / or 106 can in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that various different system configurations are possible, which can differ from the system 100. Thus, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0032] The user can use the client device 101, 102, 103, 104, 105, and / or 106 to receive the plurality of prediction boxes obtained according to the image detection method of the present disclosure. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface. Although Figure 1 Only six client devices are depicted, but one of skill in the art will appreciate that any number of client devices can be supported by the present disclosure.

[0033] The client devices 101, 102, 103, 104, 105, and / or 106 can include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service kiosk devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices can run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices can include cellular telephones, smartphones, tablet computers, personal digital assistants (PDAs), and the like. Wearable devices can include head-mounted displays (such as smart glasses) and other devices. Gaming systems can include various handheld gaming devices, Internet-enabled gaming devices, and the like. The client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0034] Network(s) 110 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of available protocols, including without limitation TCP / IP, SNA, IPX, etc. As examples, one or more of networks 110 can be a LAN, an Ethernet network, a Token Ring network, a WAN, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a Bluetooth network, a WIFI network), and / or any combination of these and / or other networks.

[0035] Server 120 can include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers), server clusters, or any other appropriate arrangement and / or combination. Server 120 can include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 can be adapted to run one or more services or software applications described below.

[0036] Computing units in server 120 can run one or more operating systems, including any of the operating systems described above, as well as any commercially available server operating systems. Server 120 can also be running one or more additional server applications, and / or any of a variety of hosted services, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0037] In some implementations, server 120 can include one or more applications to analyze and consolidate data feeds and / or event updates from users of client devices 101, 102, 103, 104, 105, and / or 106. Server 120 can also include one or more applications to display the data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and / or 106.

[0038] In some embodiments, the server 120 can be a server of a distributed system, or a server combined with a blockchain. The server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS, Virtual Private Server) services.

[0039] The system 100 can also include one or more databases 130. In certain embodiments, these databases can be used to store data and other information. For example, one or more of the databases 130 can be used to store information such as audio files and video files. The databases 130 can reside at various locations. For example, databases used by the server 120 can be local to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network- or application-specific connection. The databases 130 can be of different types. In certain embodiments, databases used by the server 120 can be, for example, relational databases. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0040] In certain embodiments, one or more of the databases 130 can also be used by applications to store application data. Databases used by applications can be databases of different types, such as key-value stores, object stores, or regular stores supported by file systems.

[0041] Figure 1 The system 100 can be configured and operated in various ways to enable the application of various methods and apparatuses described according to the present disclosure.

[0042] In the related art, a predetermined number of prior boxes (anchors) corresponding to objects in a to-be-detected image are designed according to priori, and target detection is performed on the image according to the predetermined number of prior boxes to obtain each object in the image corresponding to the predetermined number of prior boxes. When the scale distribution of the object is wide, the object in the image without a corresponding prior box is often unable to be detected due to the limited number of prior boxes.

[0043] According to an aspect of the present disclosure, an image detection method is provided. As shown in Figure 2 The image detection method 200 according to some embodiments of the present disclosure includes:

[0044] Step S210: obtaining a target image to be detected, the target image including a plurality of objects;

[0045] Step S220: performing feature extraction on the target image to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of prior boxes corresponding to a plurality of scales;

[0046] Step S230: for each of a plurality of features corresponding to the plurality of feature scales, obtaining sub-features corresponding to each of a plurality of prior boxes corresponding to the feature;

[0047] Step S240: obtaining a plurality of target sub-features from a plurality of sub-features corresponding to the plurality of features; and

[0048] Step S250: obtaining a plurality of prediction boxes of the target image based on the plurality of target sub-features, each of the plurality of prediction boxes indicating one of the plurality of objects.

[0049] For a target image to be detected, features are extracted to obtain features of each of a plurality of feature scales, and since each of the plurality of feature scales corresponds to a plurality of prior boxes, in the prediction process, a plurality of prior boxes corresponding to a plurality of feature scales are screened to obtain sub-features of prior boxes for which prediction is to be performed to obtain corresponding prediction boxes, so that in the image detection process, sub-features of suitable prior boxes in a plurality of prior boxes corresponding to a plurality of features are dynamically selected for prediction, thereby expanding the range of scales of objects detected in image detection, so that target detection can be performed on objects of more scales.

[0050] In some embodiments, in step S210, the target image can be any image to be detected, such as an image obtained from a camera. The objects contained in the target image can be any object, for example, when the target image is an image obtained from a vehicle-mounted camera, the plurality of objects contained therein can include vehicles, pedestrians, traffic lights, zebra crossings, lane lines, buildings, traffic cones, etc., which are not limited herein.

[0051] In some embodiments, in step S220, a feature extraction network is used to perform feature extraction on the target image. In some embodiments, the feature extraction network can be a backbone network based on resnet, densenet, darknet, etc.

[0052] The feature extraction network includes a plurality of feature extraction layers that obtain features corresponding to each of a plurality of feature scales by down-sampling the target image. The higher the down-sampling factor, the smaller the feature scale.

[0053] It can be understood that the higher the down-sampling multiple is, the larger the receptive field of the obtained feature is, i.e., the larger the scale of the object that can be detected based on the feature is; on the contrary, the smaller the down-sampling multiple is, the smaller the scale of the object that can be detected based on the feature is.

[0054] According to an embodiment of the present disclosure, each of the plurality of feature scales corresponds to a plurality of prior boxes. The plurality of prior boxes can be obtained by clustering different objects in advance. For example, for different objects, clustering is performed based on the respective scales to obtain a plurality of prior boxes with aspect ratios conforming to the aspect ratios of the respective objects.

[0055] In some embodiments, the number of the plurality of prior boxes corresponding to each of the plurality of feature scales is as large as possible to cover as many scales of objects as possible.

[0056] In some embodiments, the plurality of prior boxes corresponding to each of the plurality of feature scales is the same. That is, the plurality of scales of the plurality of prior boxes corresponding to each of the plurality of feature scales is the same.

[0057] In some embodiments, for a first feature scale of the plurality of feature scales and a second feature scale larger than the first feature scale, the scale of each of the plurality of prior boxes corresponding to the first feature scale is smaller than the scale of each of the plurality of prior boxes corresponding to the second feature scale.

[0058] For example, for a first feature scale with a feature scale of 128*128, the plurality of prior boxes corresponding thereto can be prior boxes with a height of any value in 70-110 and a width of any value in 70-110. For a second feature scale with a feature scale of 64*64, the plurality of prior boxes corresponding thereto can be prior boxes with a height of any value in 30-50 and a width of any value in 30-50.

[0059] For a feature with a larger feature scale, the receptive field thereof is smaller, and the scale of the prior box corresponding thereto is set to be small, so that the feature can be used to detect objects with a smaller scale. For a feature with a smaller feature scale, the receptive field thereof is larger, and the scale of the prior box corresponding thereto is set to be large, so that the feature can be used to detect objects with a larger scale. In this way, the feature with a smaller feature scale is prevented from being used to detect an object with a smaller scale and failing to accurately detect the object, and the accuracy of the detection result is improved.

[0060] In some embodiments, in step S230, for each feature, a sliding window based traversal is performed to obtain a sub-feature of the feature corresponding to each of the plurality of prior boxes. In some embodiments, in step S230, for each of the plurality of features, the feature is input into the holistic prediction head to obtain a sub-feature of the feature corresponding to each of the plurality of prior boxes.

[0061] For example, for a feature with a feature scale of 128*128, by inputting the feature into the holistic prediction head, the holistic prediction head matches each of the plurality of prior boxes corresponding to the feature based on each of a plurality of points on the feature to obtain a sub-feature of each of the plurality of prior boxes corresponding to the feature at each of the plurality of points; for each of the plurality of prior boxes, a sub-feature of the feature corresponding to the prior box is obtained from the plurality of sub-features of the feature corresponding to the prior box at the plurality of points. The sub-feature of the feature corresponding to the prior box is the sub-feature with the highest confidence from the plurality of sub-features of the feature corresponding to the prior box at the plurality of points.

[0062] In some embodiments, as shown in FIG. 2B, step S240 of obtaining a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features comprises: Figure 3

[0063] Step S310: for each of the plurality of features, obtain a prediction probability corresponding to each of the plurality of sub-features corresponding to the feature, the prediction probability indicating a degree of importance of the sub-feature in the plurality of sub-features; and

[0064] Step S320: based on the prediction probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features, obtain the plurality of target sub-features.

[0065] By obtaining the prediction probability of the plurality of sub-features corresponding to each of the features, the prediction probability indicating the degree of importance of the sub-feature in the plurality of sub-features, the obtained target sub-features are sub-features with higher importance, improving the prediction accuracy.

[0066] In some embodiments, in step S310, for each of the plurality of features, based on the plurality of sub-features of the feature, a fusion feature is obtained, and based on the fusion feature, a probability corresponding to each of the plurality of sub-features is obtained.

[0067] ​In some embodiments, in the process of obtaining the fusion feature based on the plurality of sub-features of the feature, the plurality of sub-features can be scaled correspondingly to obtain a plurality of features in the same scale, and the plurality of features in the same scale can be spliced in the channel direction to obtain the fusion feature.

[0068] In some embodiments, for each of the plurality of features, the feature is first fused with the plurality of sub-features corresponding to the plurality of features to obtain a plurality of fused features, and the plurality of fused features are spliced in the channel direction to obtain the fusion feature.

[0069] In some embodiments, in step S310, for each of the plurality of features, the plurality of sub-features corresponding to the feature are input into the scale perception module to obtain a probability corresponding to each of the plurality of sub-features.

[0070] In some embodiments, the scale perception module includes a global average pooling network and a fully connected layer, and in step S310, the fusion feature obtained based on the plurality of sub-features corresponding to the feature is globally averaged pooled by the global average pooling network to obtain a pooled feature, and the pooled feature is input into the fully connected layer to obtain a probability corresponding to each of the plurality of sub-features.

[0071] In some embodiments, the obtaining the plurality of target sub-features based on the predicted probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features includes:

[0072] For each of the plurality of features, at least one first target sub-feature corresponding to the feature is obtained from the plurality of sub-features corresponding to the feature, wherein the predicted probability of each of the at least one first target sub-feature is greater than the predicted probability of a first sub-feature in the plurality of sub-features, and the first sub-feature is different from each of the at least one first target sub-feature; and

[0073] The plurality of target sub-features are obtained based on the at least one first target sub-feature corresponding to each of the plurality of features.

[0074] From the plurality of sub-features corresponding to each feature, at least one first target sub-feature with high importance is obtained, and the plurality of target sub-features are obtained based on the at least one first target sub-feature, so that the obtained plurality of target sub-features are obtained based on the plurality of prior boxes corresponding to each feature, and can cover the receptive field of each feature, so that the size range of the object corresponding to the obtained detection box is as wide as possible.

[0075] In some embodiments, for each of the plurality of features, one or more first sub-features corresponding to the plurality of sub-features are obtained, wherein a prediction probability of each of the one or more first sub-features is greater than a preset threshold; and the one or more first sub-features corresponding to each of the plurality of features are determined as a plurality of first target sub-features corresponding to the feature.

[0076] In some embodiments, for each of the plurality of features, a preset number of second sub-features corresponding to the plurality of sub-features are obtained, wherein a prediction probability of each of the preset number of second sub-features is greater than a prediction probability of a sub-feature corresponding to the feature; and the preset number of second sub-features corresponding to each of the plurality of features are determined as a plurality of first target sub-features corresponding to the feature.

[0077] In some embodiments, in step S250, each of the plurality of target sub-features is input into the prediction head for prediction, so that the prediction head performs classification and regression for each of the plurality of target sub-features, thereby obtaining a category and a detection box corresponding to each of the plurality of target sub-features.

[0078] In some embodiments, the plurality of target sub-features includes at least one first target sub-feature corresponding to each of the plurality of features, as shown in Figure 4 As shown, step S250, based on the plurality of target sub-features, obtaining a plurality of prediction boxes of the target image includes:

[0079] Step S410: for each of the plurality of features, based on at least one first target sub-feature corresponding to the feature, obtaining a plurality of first prediction boxes corresponding to the feature; and

[0080] Step S420: based on the plurality of first prediction boxes corresponding to each of the plurality of features, obtaining the plurality of prediction boxes.

[0081] By performing regression and classification on at least one first target sub-feature corresponding to each of the plurality of features in the plurality of target sub-features, a plurality of first prediction boxes corresponding to the feature are obtained, which reduces the amount of calculation compared to performing regression and classification on each of the plurality of target sub-features respectively.

[0082] In some embodiments, for each of the plurality of features, at least one first target sub-feature corresponding to the feature in the plurality of target sub-features is input into the prediction head to obtain a plurality of first prediction boxes corresponding to the feature.

[0083] In some embodiments, the plurality of first prediction boxes corresponding to each of the plurality of features are used as the plurality of prediction boxes corresponding to the target image.

[0084] Referring to Figure 5, which shows a schematic diagram of the processing flow of the image detection method according to some embodiments of the present disclosure.

[0085] like Figure 5 As shown, first, a feature extraction network 510 including multiple feature extraction layers is used to extract features from a target image 500 to obtain multiple features 501, 502, and 503 corresponding to multiple feature scales. Then, each of the multiple features 501, 502, and 503 is input into a corresponding overall prediction head 520 to obtain multiple sub-features corresponding to the feature, and the multiple sub-features are input into a scale perception module 530 to obtain prediction probabilities corresponding to the multiple sub-features. Then, for each of the multiple features 501, 502, and 503, based on the multiple sub-features corresponding to the feature and the prediction probabilities corresponding to the multiple sub-features, an optimal selection module 540 is used to screen target sub-features to obtain at least one first target sub-feature corresponding to each feature. Finally, for each of the multiple features 501, 502, and 503, a prediction head 550 is used to perform classification (cls) and regression (Reg) based on the at least one first target sub-feature corresponding to the feature to obtain multiple first detection boxes corresponding to the feature.

[0086] It can be understood that in the above processing flow, the image detection model composed of the feature extraction network 510 and the overall prediction head 520 corresponding to each feature, the scale perception module 530, the optimal selection module 540, and the prediction head 550 realizes the detection of the target image 500. During the training process, similar to the prediction process, a training image including multiple objects is input into the image detection model, and multiple prediction boxes are output by the feature extraction network 510 and the overall prediction head 520 corresponding to each feature, the scale perception module 530, the optimal selection module 540, and the prediction head 550. After the loss is calculated based on the multiple prediction boxes and the labeled boxes for the multiple objects in the training image, the model parameters are updated based on the loss, thereby realizing model training.

[0087] In some embodiments, as Figure 6 As shown, step S240, obtaining the multiple target sub-features based on the predicted probability corresponding to each of the multiple sub-features corresponding to each of the multiple features, includes:

[0088] Step S610: obtaining a prediction probability of each of the multiple features based on the prediction probability corresponding to each of the multiple sub-features corresponding to each of the multiple features, where the prediction probability indicates the importance of the feature among the multiple features;

[0089] Step S620: obtaining at least one target feature in the plurality of features, each of the at least one target feature having a prediction probability greater than a prediction probability of a first feature in the plurality of features, the first feature being different from each of the at least one target feature; and

[0090] Step S630: obtaining the plurality of target sub-features based on the plurality of sub-features corresponding to each of the at least one target feature.

[0091] Since the objects in the same image are limited, the scale distribution range of the plurality of objects is limited, in order to avoid calculating more redundant sub-features, by obtaining the plurality of target features in the plurality of features, i.e. based on the features extracted by the priority feature extraction layer, the target sub-features are obtained, the calculation amount is reduced, and the consumption of calculation resources is reduced.

[0092] In some embodiments, in step S610, obtaining the prediction probability of each of the plurality of features based on the prediction probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features comprises:

[0093] For each of the plurality of features, obtaining at least one second target sub-feature in the plurality of sub-features corresponding to the feature, each of the at least one second target sub-feature having a corresponding prediction probability greater than a prediction probability of a second sub-feature in the plurality of sub-features, the second sub-feature being different from each of the at least one second target sub-feature; and

[0094] For each of the plurality of features, obtaining the prediction probability corresponding to the feature based on the prediction probability corresponding to each of the at least one second target sub-feature corresponding to the feature.

[0095] From the plurality of sub-features corresponding to each feature, at least one second target sub-feature with high importance is obtained, and the prediction probability of each of the at least one second target sub-feature is obtained, so that the prediction probability corresponding to the feature is obtained based on the at least one second target sub-feature with high importance in the plurality of sub-features corresponding to the feature, which indicates the probability of obtaining an accurate prediction box based on the feature, so that the plurality of target features obtained according to the prediction probability of each of the plurality of features can obtain an accurate prediction box, and the accuracy of the plurality of prediction boxes of the obtained target image is improved.

[0096] In some embodiments, the at least one second target sub-feature of each of the plurality of features is determined by obtaining a preset number of sub-features of the plurality of sub-features corresponding to the feature, and determining the preset number of sub-features as the at least one second target sub-feature corresponding to the feature.

[0097] In some embodiments, the at least one second target sub-feature of each of the plurality of features is determined by obtaining at least one sub-feature of the plurality of sub-features corresponding to the feature, wherein a prediction probability of the at least one sub-feature is greater than a probability threshold, and determining the at least one sub-feature as the at least one second target sub-feature corresponding to the feature.

[0098] In some embodiments, as shown in FIG. 25, step S250, the plurality of target sub-features includes a plurality of sub-features corresponding to each of the at least one target feature, and the obtaining the plurality of prediction boxes of the target image based on the plurality of target sub-features includes: Figure 7

[0099] Step S710: for each of the at least one target feature, obtaining a plurality of second prediction boxes corresponding to the target feature based on a plurality of sub-features corresponding to the target feature; and

[0100] Step S720: obtaining the plurality of prediction boxes based on the plurality of second prediction boxes corresponding to each of the at least one target feature.

[0101] By performing regression and classification on the plurality of sub-features of each feature, the plurality of prediction boxes corresponding to the feature are obtained, which reduces the amount of calculation and reduces the consumption of computing resources.

[0102] In some embodiments, for each of the plurality of target features, the plurality of sub-features corresponding to the target feature is input into a prediction head to obtain a plurality of second prediction boxes corresponding to the target feature.

[0103] In some embodiments, the plurality of second prediction boxes corresponding to each of the plurality of target features is taken as the plurality of prediction boxes corresponding to the target image.

[0104] According to another aspect of the present disclosure, an image detection device is also provided. As shown in FIG. 7, Figure 8 ​As shown, the apparatus 800 includes: a target image acquisition unit 810 configured to obtain a target image to be detected, the target image including a plurality of objects; a feature extraction unit 820 configured to perform feature extraction on the target image to be detected to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of prior boxes corresponding to a plurality of scales; a sub-feature acquisition unit 830 configured to, for each of a plurality of features corresponding to the plurality of feature scales, obtain sub-features corresponding to each of a plurality of prior boxes corresponding to the feature; a target sub-feature acquisition unit 840 configured to obtain a plurality of target sub-features from a plurality of sub-features corresponding to the plurality of features; and a prediction box acquisition unit 850 configured to obtain a plurality of prediction boxes of the target image based on the plurality of target sub-features, each of the plurality of prediction boxes indicating one of the plurality of objects.

[0105] In some embodiments, for a first feature scale of the plurality of feature scales and a second feature scale greater than the first feature scale, a scale of each of a plurality of prior boxes corresponding to the first feature scale is smaller than a scale of each of a plurality of prior boxes corresponding to the second feature scale.

[0106] In some embodiments, the target sub-feature acquisition unit includes: a prediction probability acquisition unit configured to, for each of the plurality of features, obtain a prediction probability corresponding to each of a plurality of sub-features corresponding to the feature, the prediction probability indicating a degree of importance of the sub-feature in the plurality of sub-features; and a first acquisition unit configured to obtain the plurality of target sub-features based on the prediction probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features.

[0107] In some embodiments, the first acquisition unit includes: a first acquisition sub-unit configured to, for each of the plurality of features, obtain at least one first target sub-feature corresponding to the feature from a plurality of sub-features corresponding to the feature, wherein a prediction probability of each of the at least one first target sub-feature is greater than a prediction probability of a first sub-feature of the plurality of sub-features, the first sub-feature being different from each of the at least one first target sub-feature; and a second acquisition sub-unit configured to obtain the plurality of target sub-features based on the at least one first target sub-feature corresponding to each of the plurality of features.

[0108] In some embodiments, the prediction box obtaining unit comprises: a first prediction unit, configured to, for each of the plurality of features, obtain a plurality of first prediction boxes corresponding to the feature based on at least one first target sub-feature corresponding to the feature; and a first prediction box obtaining sub-unit, configured to obtain the plurality of prediction boxes based on the plurality of first prediction boxes corresponding to each of the plurality of features.

[0109] In some embodiments, the first obtaining unit comprises: a second prediction probability obtaining unit, configured to obtain a prediction probability of each of the plurality of features based on a prediction probability corresponding to each of a plurality of sub-features corresponding to the feature, the prediction probability indicating a degree of importance of the feature in the plurality of features; a third obtaining sub-unit, configured to obtain at least one target feature in the plurality of features, a prediction probability of each of the at least one target feature being greater than a prediction probability of a first feature in the plurality of features, the first feature being different from each of the at least one target feature; and a fourth obtaining sub-unit, configured to obtain the plurality of target sub-features based on a plurality of sub-features corresponding to each of the at least one target feature.

[0110] In some embodiments, the second prediction probability obtaining unit comprises: a fifth obtaining sub-unit, configured to, for each of the plurality of features, obtain at least one second target sub-feature in a plurality of sub-features corresponding to the feature, a prediction probability corresponding to each of the at least one second target sub-feature being greater than a prediction probability of a second sub-feature in the plurality of sub-features, the second sub-feature being different from each of the at least one second target sub-feature; and a fifth obtaining sub-unit, configured to, for each of the plurality of features, obtain a prediction probability corresponding to the feature based on the prediction probability corresponding to each of the at least one second target sub-feature.

[0111] In some embodiments, the prediction box obtaining unit comprises: a second prediction unit, configured to, for each of the at least one target feature, obtain a plurality of second prediction boxes corresponding to the target feature based on a plurality of sub-features corresponding to the target feature; and a second prediction box obtaining sub-unit, configured to obtain the plurality of prediction boxes based on the plurality of second prediction boxes corresponding to each of the at least one target feature.

[0112] According to embodiments of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0113] Reference Figure 9, a block diagram of an electronic device 900 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0114] like Figure 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0115] Multiple components within electronic device 900 are connected to I / O interface 905, including an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. Input unit 906 can be any type of device capable of inputting information into electronic device 900. Input unit 906 can receive input numeric or character information and generate key signal input related to user settings and / or function control of the electronic device. It may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 907 can be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 908 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 909 allows electronic device 900 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks. It may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0116] The computing unit 901 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above, such as the method 200. For example, in some embodiments, the method 200 can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded onto the RAM 903 and executed by the computing unit 901, one or more steps of the method 200 described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the method 200 by any other appropriate means, such as by means of firmware.

[0117] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0118] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0119] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0121] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0122] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0123] It should be understood that the various forms of flow illustrated above can be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technology disclosed in the present disclosure can be achieved, which is not limited herein.

[0124] While embodiments or examples of the present disclosure have been described with reference to the drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but is only limited by the claims and their equivalents. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. An image detection method, comprising: obtaining a target image to be detected, wherein the target image includes a plurality of objects; performing feature extraction on the target image to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of a priori boxes, the plurality of a priori boxes correspond to a plurality of scales, and the plurality of scales corresponding to the plurality of a priori boxes are determined by clustering based on respective scales of a plurality of objects; For each of the multiple features corresponding to the multiple feature scales, obtaining a sub-feature corresponding to each of the multiple prior frames; Obtaining a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features; and Based on the multiple target sub-features, a plurality of prediction boxes of the target image are obtained, each of the plurality of prediction boxes indicating one object among the plurality of objects.

2. The method according to claim 1, wherein The obtaining of a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features comprises: For each of the plurality of features, obtaining a prediction probability corresponding to each of the plurality of sub-features corresponding to the feature, the prediction probability indicating the importance of the sub-feature among the plurality of sub-features; and The multiple target sub-features are obtained based on the predicted probability corresponding to each of the multiple sub-features corresponding to each of the multiple features.

3. The method according to claim 2, wherein The obtaining of the plurality of target sub-features based on the predicted probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features includes: For each of the multiple features, obtaining at least one first target sub-feature corresponding to the feature from the multiple sub-features corresponding to the feature, wherein a predicted probability of each of the at least one first target sub-feature is greater than a predicted probability of a first sub-feature in the multiple sub-features, and the first sub-feature is different from each of the at least one first target sub-feature; and The plurality of target sub-features are obtained based on at least one first target sub-feature corresponding to each of the plurality of features.

4. The method according to claim 3, wherein The obtaining of a plurality of prediction frames of the target image based on the plurality of target sub-features includes: For each of the multiple features, based on at least one first target sub-feature corresponding to the feature, obtain multiple first prediction boxes corresponding to the feature; and The multiple prediction boxes are obtained based on the multiple first prediction boxes corresponding to each feature of the multiple features.

5. The method according to claim 2, wherein: The obtaining of the plurality of target sub-features based on the predicted probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features includes: Obtaining a predicted probability for each of the plurality of features based on a predicted probability corresponding to each of the plurality of sub-features corresponding to each of the plurality of features, the predicted probability indicating an importance of the feature among the plurality of features; obtaining at least one target feature from the plurality of features, wherein a predicted probability of each target feature from the at least one target feature is greater than a predicted probability of a first feature from the plurality of features, the first feature being distinct from each target feature from the at least one target feature; and The multiple target sub-features are obtained based on the multiple sub-features corresponding to each target feature of the at least one target feature.

6. The method according to claim 5, wherein: The obtaining, based on the prediction probability corresponding to each of the multiple sub-features corresponding to each of the multiple features, the prediction probability of each of the multiple features includes: For each of the plurality of features, obtaining at least one second target sub-feature from among the plurality of sub-features corresponding to the feature, wherein a corresponding prediction probability of each of the at least one second target sub-feature is greater than a prediction probability of a second sub-feature from among the plurality of sub-features, and the second sub-feature is distinct from each of the at least one second target sub-feature; and Based on the predicted probability corresponding to each second target sub-feature of the at least one second target sub-feature, the predicted probability corresponding to the feature is obtained.

7. The method according to claim 5, wherein The multiple target sub-features include multiple sub-features corresponding to each target feature of the at least one target feature, and obtaining multiple prediction frames of the target image based on the multiple target sub-features includes: For each target feature of the at least one target feature, based on a plurality of sub-features corresponding to the target feature, obtaining a plurality of second prediction boxes corresponding to the target feature; and The multiple prediction boxes are obtained based on the multiple second prediction boxes corresponding to each target feature of the at least one target feature.

8. An image detection device, comprising: a target image acquisition unit, configured to obtain a target image to be detected, wherein the target image includes a plurality of objects; a feature extraction unit configured to perform feature extraction on the target image to be detected to obtain features corresponding to each of a plurality of feature scales, wherein each of the plurality of feature scales corresponds to a plurality of a priori boxes, the plurality of a priori boxes correspond to a plurality of scales, and the plurality of scales corresponding to the plurality of a priori boxes are determined by clustering based on the respective scales of a plurality of objects; a sub-feature obtaining unit configured to obtain, for each of the multiple features corresponding to the multiple feature scales, a sub-feature corresponding to each of the multiple prior frames; a target sub-feature acquisition unit configured to obtain a plurality of target sub-features from the plurality of sub-features corresponding to the plurality of features; and The prediction frame acquisition unit is configured to obtain a plurality of prediction frames of the target image based on the plurality of target sub-features, each of the plurality of prediction frames indicating one object among the plurality of objects.

9. The device according to claim 8, wherein The target sub-feature acquisition unit includes: a prediction probability acquisition unit configured to obtain, for each feature of the plurality of features, a prediction probability corresponding to each of the plurality of sub-features corresponding to the feature, the prediction probability indicating a degree of importance of the sub-feature among the plurality of sub-features; and The first acquisition unit is configured to obtain the multiple target sub-features based on the prediction probability corresponding to each of the multiple sub-features corresponding to each of the multiple features.

10. The device according to claim 9, wherein The first acquiring unit includes: a first acquiring subunit configured to, for each feature of the plurality of features, obtain, from the plurality of subfeatures corresponding to the feature, at least one first target subfeature corresponding to the feature, wherein a predicted probability of each of the at least one first target subfeature is greater than a predicted probability of a first subfeature of the plurality of subfeatures, and the first subfeature is distinguishable from each of the at least one first target subfeature; and The second acquisition subunit is configured to obtain the multiple target sub-features based on at least one first target sub-feature corresponding to each feature of the multiple features.

11. The device according to claim 10, wherein The prediction frame acquisition unit includes: A first prediction unit is configured to obtain, for each feature of the plurality of features, a plurality of first prediction boxes corresponding to the feature based on at least one first target sub-feature corresponding to the feature; and The first prediction box acquisition subunit is configured to obtain the multiple prediction boxes based on the multiple first prediction boxes corresponding to each feature of the multiple features.

12. The device according to claim 9, wherein The first acquiring unit includes: a second prediction probability acquisition unit configured to obtain a prediction probability of each of the multiple features based on a prediction probability corresponding to each of the multiple sub-features corresponding to each of the multiple features, the prediction probability indicating a degree of importance of the feature among the multiple features; a third acquisition subunit configured to obtain at least one target feature from the plurality of features, wherein a prediction probability of each target feature from the at least one target feature is greater than a prediction probability of a first feature from the plurality of features, the first feature being distinguishable from each target feature from the at least one target feature; and The fourth acquisition subunit is configured to obtain the multiple target sub-features based on the multiple sub-features corresponding to each target feature of the at least one target feature.

13. The device according to claim 12, wherein The second prediction probability acquisition unit includes: a fifth acquisition subunit configured to obtain, for each feature of the plurality of features, at least one second target subfeature from among the plurality of subfeatures corresponding to the feature, wherein a corresponding prediction probability of each of the at least one second target subfeature is greater than a prediction probability of a second subfeature from among the plurality of subfeatures, and the second subfeature is different from each of the at least one second target subfeature; and The fifth acquisition subunit is configured to obtain, for each of the multiple features, a predicted probability corresponding to the feature based on the predicted probability corresponding to each second target sub-feature of the at least one second target sub-feature.

14. The device according to claim 13, wherein The prediction frame acquisition unit includes: A second prediction unit is configured to obtain, for each target feature of the at least one target feature, a plurality of second prediction boxes corresponding to the target feature based on a plurality of sub-features corresponding to the target feature; and The second prediction frame acquisition subunit is configured to obtain the multiple prediction frames based on the multiple second prediction frames corresponding to each target feature of the at least one target feature.

15. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images