Image detection method, processing core, electronic device and computer readable medium

Abstract

This disclosure provides an image detection method, comprising: performing image detection on at least one image to be processed using a first model and a second model; when the second model performs image detection, determining an evaluation value corresponding to the first detection result in response to a first detection result obtained by the first model; and stopping the detection of at least one image to be processed by the second model and outputting the first detection result in response to a situation where the evaluation value corresponding to the first detection result is greater than or equal to a preset output threshold. This disclosure also provides a processing core, an electronic device, and a computer-readable medium.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image detection method, processing core, electronic device, and computer-readable medium. Background Technology

[0002] With the development of computer technology and neural network technology, image detection and recognition algorithms have been widely used in various fields. Correspondingly, facing diverse detection and recognition task requirements, the network models used for detection and recognition have become increasingly diverse and complex. However, in the actual execution of detection and recognition tasks, it is difficult to balance detection and recognition accuracy with detection and recognition speed, and it is impossible to simultaneously guarantee the reliability of detection and recognition results and the processing speed of the detection and recognition task. Summary of the Invention

[0003] This disclosure provides an image detection method, a processing kernel, an electronic device, and a computer-readable medium.

[0004] In a first aspect, this disclosure provides an image detection method, including:

[0005] Image detection is performed on at least one image to be processed using a first model and a second model, wherein the first model and the second model start image detection simultaneously;

[0006] When the second model performs image detection, in response to the first detection result obtained by the first model, an evaluation value corresponding to the first detection result is determined;

[0007] In response to the situation where the evaluation value corresponding to the first detection result is greater than or equal to a preset output threshold, the second model stops detecting the at least one image to be processed and outputs the first detection result.

[0008] Secondly, this disclosure provides a processing core for use in many-core systems, the processing core comprising:

[0009] One or more processing units;

[0010] A storage unit stores one or more programs, which, when executed by the one or more processing units, enable the one or more processing units to implement the image detection method described above.

[0011] Thirdly, this disclosure provides an electronic device, including:

[0012] Multiple processing cores; and

[0013] The on-chip network is configured to interact with data between the multiple processing cores and external data;

[0014] One or more processing cores store one or more instructions, and the one or more instructions are executed by one or more processing cores to enable one or more processing cores to perform the image detection method described above.

[0015] Fourthly, this disclosure provides an electronic device, including:

[0016] one or more processors;

[0017] a memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image detection method described above.

[0019] Fifthly, this disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processing core, implements the steps of the image detection method described above.

[0020] The image detection method, processing kernel, electronic device, and computer-readable medium provided in this disclosure perform image detection by using multiple models in parallel. When the detection result obtained earlier meets the corresponding conditions, the image detection tasks of other models are stopped and the detection result is output. This achieves the goal of obtaining the detection result quickly and with less resource consumption while ensuring the reliability of the detection result, thereby improving the efficiency of image detection.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0023] Figure 1 A flowchart of an image detection method provided in this disclosure embodiment;

[0024] Figure 2 Flowchart of another image detection method provided in the embodiments of this disclosure

[0025] Figure 3 A flowchart of yet another image detection method provided in this disclosure embodiment;

[0026] Figure 4This is a flowchart of a specific implementation method of step S02 in this disclosure embodiment;

[0027] Figure 5 This is a flowchart of a specific implementation method of step S2 in this disclosure embodiment;

[0028] Figure 6 This is a flowchart of a specific implementation method of step S1 in this disclosure embodiment;

[0029] Figure 7 A flowchart illustrating yet another image detection method provided in this disclosure embodiment;

[0030] Figure 8 A block diagram of a processing core provided in an embodiment of this disclosure;

[0031] Figure 9 A block diagram of an electronic device provided in an embodiment of this disclosure;

[0032] Figure 10 A block diagram of another electronic device provided in an embodiment of this disclosure;

[0033] Figure 11 This is a block diagram illustrating the composition of a computer-readable medium provided in an embodiment of the present disclosure. Detailed Implementation

[0034] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0036] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0037] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0038] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0039] Figure 1 This is a flowchart illustrating an image detection method provided in an embodiment of this disclosure. Figure 1 As shown, the method includes:

[0040] Step S1: Perform image detection on at least one image to be processed using the first model and the second model.

[0041] In this model, the first model and the second model start image detection simultaneously, meaning that the first model and the second model start executing the image detection task at the same time. The first model and the second model are neural network models used to implement specific image detection algorithms.

[0042] Image detection algorithms can be applied to image classification, object detection, facial recognition, and pose recognition. In some embodiments, step S1, which involves performing image detection on at least one image to be processed using a first model and a second model, includes: performing object detection or object recognition on at least one image to be processed using the first model and the second model. That is, image detection includes either object detection or object recognition. Specifically, object detection, also known as object discovery, is used to detect the presence of a target object; object recognition is used to identify the category of a target object in an image or to classify the target object.

[0043] The first model and the second model can be two models with different neural network structures, or two models with different algorithms, or two models with the same algorithm but different accuracy. It should be noted that the above description is only one specific optional implementation method, used to ensure that the first model and the second model obtain detection results sequentially. Other model selection methods that can achieve this effect are also applicable to the technical solution of this application.

[0044] In step S1, the first model and the second model are used in parallel to perform image detection on the image to be processed; in some embodiments, multiple identical first models and multiple identical second models are used in parallel to perform image detection on the image to be processed.

[0045] In some embodiments, the detection accuracy of the first model is lower than that of the second model. When the detection accuracy is lower, the detection time is correspondingly shorter; that is, the first model is a low-precision, low-time-consuming detection model, and the second model is a high-precision, high-time-consuming detection model.

[0046] Step S2: When the second model performs image detection, in response to the first detection result obtained by the first model, determine the evaluation value corresponding to the first detection result.

[0047] The evaluation value can be the probability value in the corresponding test result, or the confidence level of the test result, or the accuracy of the test result, etc.

[0048] Step S3: In response to the situation where the evaluation value corresponding to the first detection result is greater than or equal to the preset output threshold, stop the second model from detecting the image of the at least one image to be processed, and output the first detection result.

[0049] The output threshold corresponds to the evaluation value used, which can be a pre-configured fixed value or a dynamically changing value according to the requirements of different detection tasks.

[0050] In step S3, when the detection result obtained by one of the detection models meets the output condition, the other detection model that is still performing image detection is stopped, and the detection result that meets the output condition is output.

[0051] In some embodiments, in step S3, when the detection accuracy of the first model is less than that of the second model, if the detection result obtained by the low-precision, low-time-consuming detection model meets the requirements of the detection task, the detection of the high-precision, high-time-consuming detection model is stopped. This achieves the goal of obtaining detection or recognition results quickly and with less resource consumption while ensuring detection accuracy to a certain extent, thereby improving efficiency.

[0052] This disclosure provides an image detection method that can be used to perform image detection in parallel using multiple detection models. When the detection result obtained earlier meets the corresponding conditions, the image detection task of other detection models is stopped and the detection result is output. This method achieves the goal of obtaining the detection result quickly and with less resource consumption while ensuring the reliability of the detection result, thereby improving the efficiency of image detection.

[0053] Figure 2 A flowchart illustrating another image detection method provided in this disclosure. Figure 2 As shown, this method is based on Figure 1 A specific alternative implementation of the method shown is provided. Specifically, the method includes not only steps S1 to S3, but also step S4. Only step S4 will be described in detail below.

[0054] Step S4: In response to the situation where the evaluation value corresponding to the first detection result is less than the output threshold, wait for the second model to obtain the second detection result and output the second detection result.

[0055] In step S4, if the previously obtained detection result does not meet the corresponding output conditions, another detection model that is still performing detection continues to operate and outputs the subsequent detection result obtained by that detection model.

[0056] In some embodiments, in step S4, when the detection accuracy of the first model is less than that of the second model, if the detection result obtained by the low-precision, low-time-consuming detection model does not meet the requirements of the detection task, the operation of the high-precision, high-time-consuming detection model is maintained, and the detection result obtained by the high-precision, high-time-consuming detection model is used as the final detection result for output to ensure accuracy.

[0057] Figure 3 A flowchart illustrating yet another image detection method provided in this disclosure. (See attached flowchart.) Figure 3 As shown, this method is based on Figure 1 The illustrated method is a specific alternative implementation. Specifically, the method includes not only steps S1 to S3, but also steps S01 and S02 before step S1, which involves image detection using a first model and a second model on at least one image to be processed. Only steps S01 and S02 will be described in detail below.

[0058] Step S01: Obtain the detection accuracy requirements and time requirements corresponding to the at least one image to be processed.

[0059] In step S01, the detection accuracy requirements and detection time requirements of the detection task are obtained.

[0060] Step S02: Determine the configuration information of the first model and the second model according to the detection accuracy requirements and time requirements, and configure the first model and the second model.

[0061] The configuration information is used to determine the detection accuracy and detection time of the corresponding detection model. In some embodiments, the configuration information of each detection model can be determined comprehensively based on the detection accuracy requirements, time requirements, all images to be processed, and the target objects to be detected.

[0062] In some embodiments, the configuration information of the first model is determined according to time requirements, and the configuration information of the second model is determined according to detection accuracy requirements, so as to obtain two types of detection models that prioritize detection accuracy and those that prioritize detection time.

[0063] Figure 4 This is a flowchart illustrating a specific implementation method of step S02 in this disclosure. Figure 4 As shown, specifically, in step S02, the step of configuring the first model and the second model includes:

[0064] Step S021: Obtain multiple processing core clusters in the many-core system.

[0065] Among them, a many-core system (or many-core processor) includes multiple processing cores; a processing core, also known as a kernel or core, has functions such as application running and task processing; each processing core cluster consists of at least one processing core, and a many-core system may include multiple processing core clusters.

[0066] Step S022: Configure the first processing core cluster according to the configuration information of the first model, and configure the second processing core cluster according to the configuration information of the second model.

[0067] Among them, the first processing kernel cluster and the second processing kernel cluster belong to the plurality of processing kernel clusters; in step S022, the processing kernel clusters are configured according to the configuration information to construct the detection model, and then the detection tasks are assigned to the processing kernel clusters to perform image detection using the corresponding detection model.

[0068] In some embodiments, step S021, the step of obtaining multiple processing core clusters in the many-core system, includes: obtaining multiple first processing cores in the many-core system according to the configuration information of the first model, and forming a third processing core cluster to construct the first model; and obtaining multiple second processing cores in the many-core system according to the configuration information of the second model, and forming a fourth processing core cluster to construct the second model.

[0069] This disclosure provides an image detection method that can be used to configure multiple detection models using different processing kernel clusters in a many-core system, and to process image detection tasks using a many-core system.

[0070] Figure 5 This is a flowchart illustrating a specific implementation method of step S2 in this disclosure. Figure 5 As shown, specifically, the first detection result includes the probability of target presence and target location information; in step S2, the step of determining the evaluation value corresponding to the first detection result includes:

[0071] Step S201: Use the target probability as the evaluation value.

[0072] The obtained probability value is used as the evaluation value. When the probability value in the detection result of one of the detection models is greater than or equal to the corresponding output threshold, the detection result is immediately used as the final detection result and output. When the probability value is less than the corresponding output threshold, another detection model is used for verification, and the subsequent detection result is used as the final detection result and output.

[0073] Figure 6 This is a flowchart illustrating a specific implementation method of step S1 in this disclosure. Figure 6 As shown, specifically, step S1, the step of detecting at least one image to be processed using a first model and a second model, includes:

[0074] Step S101: Perform image detection on multiple images to be processed using the first model and the second model.

[0075] In cases where there are multiple images to be processed, taking the multiple images to be processed as a complete video stream as an example, in some embodiments, the first model and the second model can be used to perform image detection on the video stream frame by frame simultaneously, with each frame corresponding to a detection result, and the above steps are performed frame by frame; alternatively, the first model and the second model can be used to perform image detection on the entire video stream simultaneously, with each detection model ultimately obtaining a detection result, which is then used in subsequent steps.

[0076] In some embodiments, the plurality of images to be processed includes a plurality of real-time acquired images, such as monitoring images, temperature measurement images, and face images.

[0077] Figure 7 A flowchart illustrating yet another image detection method provided in this disclosure. Figure 7 As shown, this method is based on Figure 1 The illustrated method is a specific alternative implementation. Specifically, the method includes not only steps S1 to S3, but also steps S03 and S04 before step S1, which involves image detection using a first model and a second model on at least one image to be processed. Only steps S03 and S04 will be described in detail below.

[0078] Step S03: Determine the scene type corresponding to the at least one image to be processed.

[0079] Scene types can be categorized based on factors such as the type of target object to be detected, the number of target objects to be detected, the number of images to be processed, and whether the ambient light of the images to be processed is sufficient.

[0080] Step S04: Determine the corresponding first model and second model based on the scene type.

[0081] Among them, a corresponding detection model can be constructed according to the scene type; or the original detection model can be adjusted according to the scene type; or, multiple detection models with different configuration parameters can be pre-configured to be applied to different scene types, from which the corresponding first model and second model can be determined.

[0082] Figure 8 This is a block diagram illustrating the composition of a processing core provided in an embodiment of this disclosure. For example... Figure 8 As shown, this processing core, applied to many-core systems, includes:

[0083] One or more processing units 101;

[0084] The storage unit 102 stores one or more programs, which, when executed by the one or more processing units, enable the one or more processing units to implement any of the image detection methods described in the above embodiments.

[0085] Figure 9 This is a block diagram illustrating the composition of an electronic device according to an embodiment of this disclosure. Figure 9 As shown, the electronic device includes:

[0086] Multiple processing cores 201; and

[0087] The on-chip network 202 is configured to interact with data between the multiple processing cores 201 and external data;

[0088] One or more processing cores 201 store one or more instructions, which are executed by one or more processing cores 201 to enable one or more processing cores 201 to implement any of the image detection methods in the above embodiments.

[0089] Figure 10 This is a block diagram of another electronic device provided in an embodiment of this disclosure. (See diagram below.) Figure 10 As shown, the server-side device includes:

[0090] One or more processors 301;

[0091] The memory 302 stores one or more programs that, when executed by the one or more processors, enable the one or more processors to implement any of the image detection methods described in the above embodiments.

[0092] One or more I / O interfaces 303 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0093] Among them, processor 301 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 302 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 303 is connected between processor 301 and memory 302, and can realize information interaction between processor 301 and memory 302, including but not limited to data bus (Bus).

[0094] In some embodiments, the processor 301, memory 302, and I / O interface 303 are interconnected via bus 304, and thus connected to other components of the computing device.

[0095] In some embodiments, the plurality of processors 301 include a plurality of graphics processors (GPUs) configured to form a graphics processor array.

[0096] Figure 11 This is a block diagram illustrating the composition of a computer-readable medium provided in an embodiment of the present disclosure. The computer-readable medium stores a computer program, which, when executed by a processing core, implements the steps of any of the image detection methods described in the above embodiments.

[0097] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0098] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. An image detection method, wherein, include: Image detection is performed on at least one image to be processed using a first model and a second model, wherein the first model and the second model start image detection simultaneously, the neural network structures of the first model and the second model are different, or the algorithms used by the first model and the second model are different, or the algorithms used by the first model and the second model are the same but the corresponding accuracies of the algorithms are different, and the detection time of the first model is less than the detection time of the second model. When the second model performs image detection, in response to the first detection result obtained by the first model, an evaluation value corresponding to the first detection result is determined; In response to the situation where the evaluation value corresponding to the first detection result is greater than or equal to a preset output threshold, the second model stops image detection of the at least one image to be processed and outputs the first detection result.

2. The image detection method according to claim 1, wherein, The step of performing image detection on at least one image to be processed using a first model and a second model includes: The first model and the second model are used to perform target detection or target recognition on the at least one image to be processed.

3. The image detection method according to claim 1, wherein, Also includes: In response to the situation where the evaluation value corresponding to the first detection result is less than the output threshold, wait for the second model to obtain the second detection result, and then output the second detection result.

4. The image detection method according to claim 1, wherein, The detection accuracy of the first model is lower than that of the second model.

5. The image detection method according to claim 1, wherein, Before the step of performing image detection on at least one image to be processed using the first model and the second model, the method further includes: Obtain the detection accuracy requirement and time requirement corresponding to the at least one image to be processed; The configuration information of the first model and the second model is determined according to the detection accuracy requirement and the time requirement, and the first model and the second model are configured. The configuration information is used to determine the detection accuracy and detection time of the corresponding model.

6. The image detection method according to claim 5, wherein, The steps of configuring the first model and the second model include: Multiple processing core clusters in a many-core system are obtained, wherein the many-core system includes multiple processing cores, and each processing core cluster consists of at least one processing core; A first processing core cluster is configured according to the configuration information of the first model, and a second processing core cluster is configured according to the configuration information of the second model, wherein the first processing core cluster and the second processing core cluster belong to the plurality of processing core clusters.

7. The image detection method according to claim 1, wherein, The first detection result includes: target probability and target location information; The step of determining the evaluation value corresponding to the first detection result includes: The target probability is used as the evaluation value.

8. The image detection method according to claim 1, wherein, The step of detecting at least one image to be processed using a first model and a second model includes: Image detection is performed on multiple images to be processed using the first model and the second model.

9. The image detection method according to claim 8, wherein, The multiple images to be processed include multiple images acquired in real time.

10. The image detection method according to claim 1, wherein, Before the step of performing image detection on at least one image to be processed using the first model and the second model, the method further includes: Determine the scene type corresponding to the at least one image to be processed; The first model and the second model are determined according to the scenario type.

11. A processing core, applied in a many-core system, the processing core comprising: One or more processing units; A storage unit storing one or more programs, which, when executed by the one or more processing units, enable the one or more processing units to implement the image detection method as described in any one of claims 1-10.

12. An electronic device, comprising: Multiple processing cores; as well as The on-chip network is configured to interact with data between the multiple processing cores and external data; One or more processing cores store one or more instructions, and the one or more instructions are executed by one or more processing cores to enable one or more processing cores to perform the image detection method according to any one of claims 1-10.

13. An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image detection method as described in any one of claims 1-10.

14. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processing core, it implements the steps of the image detection method as described in any one of claims 1-10.

Images