Image detection method, image detection device, electronic device, and storage medium

Abstract

The present disclosure provides an image detection method, comprising: acquiring images of a target to be detected under a plurality of preset modalities; inputting images under each frame of preset modality into an image detection model corresponding to the preset modality respectively, and outputting a plurality of first image detection results; and fusing the plurality of first image detection results according to a preset strategy to obtain a target detection result for representing whether the target to be detected is a real target or an attacking target. In addition, the present disclosure also provides an image detection device, an electronic device, a readable storage medium and a computer program product.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and more specifically, to an image detection method, an image detection device, an electronic device, a readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of artificial intelligence, facial recognition systems are increasingly being used in production applications such as facial payment and facial security checks. At the same time, to prevent attacks on facial recognition systems that could lead to a series of losses, facial anti-counterfeiting technology has also been developed.

[0003] In the process of realizing the concept disclosed herein, the inventors discovered that the relevant facial anti-counterfeiting technology adopts a detection method that mainly uses the visible light modality and supplements other modalities, and performs defense based on each modality separately, failing to fully utilize the advantages of each modality and having poor robustness. Summary of the Invention

[0004] In view of this, the present disclosure provides an image detection method, an image detection device, an electronic device, a readable storage medium, and a computer program product.

[0005] One aspect of the present disclosure provides an image detection method, comprising: acquiring images of the target to be detected in multiple preset modes; inputting each frame of the image in the preset mode into an image detection model corresponding to the preset mode, and outputting multiple first image detection results; fusing the multiple first image detection results according to a preset strategy to obtain a target detection result used to characterize the target to be detected as a real target or an attack.

[0006] According to an embodiment of the present disclosure, the above-mentioned preset modality includes a depth modality; the above-mentioned method also includes: inputting the image under the above-mentioned depth modality into a depth image detection model, and outputting a second image detection result, wherein the above-mentioned second image detection result is used to characterize that the above-mentioned target to be detected is a three-dimensional target or a two-dimensional target; and when it is determined that the above-mentioned target to be detected is a three-dimensional target, inputting each frame of the image under the above-mentioned preset modality into the image detection model corresponding to the above-mentioned preset modality, and outputting multiple above-mentioned first image detection results.

[0007] According to an embodiment of the present disclosure, the method further includes: in the case where it is determined that the target to be detected is a two-dimensional target, outputting a result indicating that the target to be detected is an attack.

[0008] According to an embodiment of the present disclosure, the above method also includes: for each frame of the image under the above preset modality, determining the position of the above target to be detected in the above image; cropping the above image to a preset size with the position of the above target to be detected as the center; and inputting the cropped image into the image detection model corresponding to the above preset modality, and outputting the above first image detection result.

[0009] According to an embodiment of the present disclosure, the above-mentioned multiple first image detection results are fused according to a preset strategy to obtain a target detection result used to characterize the above-mentioned target to be detected as a real target or an attack, including: when the multiple first image detection results are consistent, determining the above-mentioned target detection result is consistent with the above-mentioned first image detection result; when there are inconsistent detection results among the multiple first image detection results, respectively determining multiple detection values ​​corresponding to the multiple first image detection results; respectively calculating the classification weight of each of the above-mentioned detection values; and selecting the above-mentioned first image detection result corresponding to the highest classification weight among the multiple classification weights as the above-mentioned target detection result.

[0010] According to an embodiment of the present disclosure, the above-mentioned image detection model includes one trained by an initial classification model, the above-mentioned initial classification model has a category label value corresponding to the image detection result, and the above-mentioned image detection model has a preset classification threshold.

[0011] According to an embodiment of the present disclosure, the above-mentioned calculation of the classification weight of each of the above-mentioned detection values ​​includes: for each first image detection result, determining the category label value corresponding to the above-mentioned first image detection result; calculating the difference between the above-mentioned detection value and the above-mentioned preset classification threshold to obtain a first difference; calculating the difference between the above-mentioned category label value and the above-mentioned preset classification threshold to obtain a second difference; and taking the absolute value of the ratio of the above-mentioned first difference and the above-mentioned second difference to obtain the above-mentioned classification weight.

[0012] According to an embodiment of the present disclosure, the above-mentioned multiple first image detection results are fused according to a preset strategy to obtain a target detection result for characterizing that the above-mentioned target to be detected is a real target or an attack, including: respectively determining multiple detection values ​​corresponding to the multiple above-mentioned first image detection results; inputting the multiple above-mentioned detection values ​​into a classification model, and outputting the above-mentioned target detection result.

[0013] According to an embodiment of the present disclosure, the multiple preset modalities include visible light modality, near infrared modality and depth modality; the images under the multiple preset modalities are used to characterize the texture information, color information and reflective properties of the target to be detected.

[0014] Another aspect of the present disclosure provides an image detection device, including a first acquisition module, a first processing module, and a second processing module, wherein: the first acquisition module is used to acquire images of the above-mentioned target to be detected in multiple preset modes; the first processing module is used to input each frame of the image in the above-mentioned preset mode into the image detection model corresponding to the above-mentioned preset mode, and output multiple first image detection results; the second processing module is used to fuse the multiple first image detection results according to a preset strategy to obtain a target detection result used to characterize the above-mentioned target to be detected as a real target or an attack.

[0015] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method described above.

[0016] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0017] Another aspect of the present disclosure provides a computer program product comprising computer executable instructions, which are used to implement the method described above when the instructions are executed.

[0018] According to the embodiments of the present disclosure, by adopting a multimodal fusion strategy, images of the target to be detected in multiple modalities are detected, and the multiple detection results are fused to obtain a detection result of whether the target to be detected is an attack or a real target. This at least partially overcomes the technical problems existing in the related art of failing to fully utilize the advantages of each modality and having poor robustness, effectively enhances the defense capability against attacks based on physical media, and improves the robustness of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0020] Figure 1 An exemplary system architecture 100 to which an image detection method according to an embodiment of the present disclosure may be applied is schematically shown.

[0021] Figure 2 The flowchart of the image detection method according to the embodiment of the present disclosure is schematically shown.

[0022] Figures 3A to 3D Schematic diagrams of depth images of targets to be detected according to embodiments of the present disclosure are schematically shown respectively.

[0023] Figure 4 The flowchart of the image detection method according to another embodiment of the present disclosure is schematically shown.

[0024] Figure 5 The block diagram schematically shows an image detection device according to an embodiment of the present disclosure.

[0025] Figure 6 A block diagram of an electronic device suitable for implementing an image detection method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0030] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0031] Facial anti-counterfeiting technology is an essential component of facial recognition systems. It verifies whether facial images captured by cameras are genuine, protecting facial recognition systems from attacks that could lead to a range of losses. In recent years, with the rapid development of deep learning technology, facial anti-counterfeiting technology has made significant progress and has been successfully applied to facial recognition systems such as facial payment, facial security checks, video surveillance, and access control gates.

[0032] In facial recognition system scenarios, physical media-based attacks are the most common. These attacks typically involve attackers presenting a face through media such as paper, screens, masks, or head models. For example, attackers can use a printed photo of a legitimate user or display a digital photo of a legitimate user on an electronic device to attack facial recognition systems. Other attacks involve using masks or 3D head models that mimic the legitimate user's appearance.

[0033] Optical modality detection is a common facial anti-counterfeiting technology. However, visible light modality is significantly affected by external environmental factors such as lighting, resulting in poor detection robustness.

[0034] In view of this, the present disclosure provides a face anti-counterfeiting method based on multimodal fusion according to the characteristics of multimodal images, so as to at least partially solve the problems existing in the related art. Specifically, the embodiments of the present disclosure provide an image detection method, an image detection device, an electronic device, a readable storage medium and a computer program product. The method includes obtaining images of a target to be detected in multiple preset modes; inputting each frame of the image in the preset mode into the image detection model corresponding to the preset mode, and outputting multiple first image detection results; fusing the multiple first image detection results according to a preset strategy to obtain a target detection result for characterizing the target to be detected as a real target or an attack.

[0035] Figure 1 The following schematically illustrates an exemplary system architecture 100 to which the image detection method according to an embodiment of the present disclosure can be applied. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0036] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 .

[0037] The terminal devices 101 , 102 , and 103 may be various electronic devices with a display screen and a camera, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0038] Various client applications may be installed on the terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. The client applications installed on the terminal devices 101, 102, and 103 may implement a face recognition function.

[0039] The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links.

[0040] The server 105 may be a server that provides various services, including but not limited to a backend server for various client applications installed on the terminal devices 101 , 102 , and 103 .

[0041] It should be noted that the image detection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image detection device provided in the embodiment of the present disclosure can generally be set in the server 105. The image detection method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image detection device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the image detection method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the image detection device provided in the embodiment of the present disclosure can also be set in the terminal devices 101, 102, or 103, or can be set in other terminal devices different from the terminal devices 101, 102, or 103.

[0042] For example, any one of the terminal devices 101, 102 or 103 (for example, terminal device 101, but not limited to this) can obtain the user's facial image through a camera and locally execute the image detection method provided by the embodiment of the present disclosure; or, the terminal device 101 can send the facial image to other terminal devices, servers, or server clusters, and the image detection method provided by the embodiment of the present disclosure can be executed by the other terminal devices, servers, or server clusters that receive the image to be processed.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0044] Figure 2 The flowchart of the image detection method according to the embodiment of the present disclosure is schematically shown.

[0045] like Figure 2 As shown, the method includes operations S201 to S203.

[0046] In operation S201 , images of a target to be detected in multiple preset modes are acquired.

[0047] In operation S202 , each frame of the image in the preset mode is input into an image detection model corresponding to the preset mode, and a plurality of first image detection results are output.

[0048] In operation S203, the plurality of first image detection results are fused according to a preset strategy to obtain a target detection result for characterizing whether the target to be detected is a real target or an attack.

[0049] According to the embodiments of the present disclosure, different targets can be detected depending on the business scenarios in which the image detection method is applied. For example, in a face recognition system, the target can be a real face or a physical medium-based attack against the face recognition system; for another example, in an object recognition system, the target can be a real object or a physical medium-based attack against the object recognition system, without limitation.

[0050] According to an embodiment of the present disclosure, the images under multiple preset modalities may include one or more frames of images under each preset modality, and the images under multiple preset modalities may be acquired by an electronic device such as a multimodal camera.

[0051] According to an embodiment of the present disclosure, images of preset modalities may include images measured based on electromagnetic waves of different wavelengths, such as visible light images and near-infrared images, and images measured based on different detection principles, such as depth images.

[0052] According to an embodiment of the present disclosure, the image detection model may be a fully trained classification model, which may use any existing model framework such as VGG Net, ResNet, ResNeXt, SE-Net, etc.

[0053] According to an embodiment of the present disclosure, the first image detection result may include a result indicating that the target to be detected in the modality is a real target or an attack.

[0054] According to an embodiment of the present disclosure, since multiple first image detection results may simultaneously contain results characterizing that the target to be detected in a certain modality is a real target and an attack, the multiple first image detection results can be fused to fully utilize the advantages of each modality and improve the robustness of the detection results.

[0055] According to an embodiment of the present disclosure, performing fusion according to a preset strategy may include selecting the most appropriate detection result from multiple first image detection results as the target detection result based on a preset selection strategy. For example, a confidence level may be set for each image detection model. If multiple first image detection results are inconsistent, the first image detection result output by the image detection model with the highest confidence level is selected as the target detection result.

[0056] According to embodiments of the present disclosure, fusion according to a preset strategy may also involve combining multiple first image detection results according to a preset operation method, and using the resulting new detection result as the target detection result. For example, a classification model may be trained that uses the detection values ​​corresponding to the multiple first image detection results as input. During use, the detection values ​​corresponding to the multiple first image detection results are sequentially combined into an input vector, which is then input into the classification model, and the resulting output result is used as the target detection result.

[0057] According to the embodiments of the present disclosure, by adopting a multimodal fusion strategy, images of the target to be detected in multiple modalities are detected, and the multiple detection results are fused to obtain a detection result of whether the target to be detected is an attack or a real target. This at least partially overcomes the technical problems existing in the related art of failing to fully utilize the advantages of each modality and having poor robustness, effectively enhances the defense capability against attacks based on physical media, and improves the robustness of the detection results.

[0058] Figures 3A to 3D Schematic diagrams of depth images of targets to be detected according to embodiments of the present disclosure are schematically shown respectively.

[0059] like Figures 3A to 3D As shown, the depth image can represent the three-dimensional information of the target to be detected, and the depth of the color of the local area indicates the relative distance between the area on the target to be detected and the depth camera that obtains the depth image.

[0060] in, Figure 3A is a real face and a depth image of a real face, Figure 3D These are mask attack and depth images of mask attack. The face parts in these two depth images have multiple areas with different depths, showing more obvious three-dimensional features. Figure 3B For screen attack and depth image of screen attack, Figure 3C These are the paper attack and the depth image of the paper attack. The face parts in these two depth images have a large area with the same depth, showing more obvious two-dimensional features.

[0061] According to an embodiment of the present disclosure, the three-dimensional or two-dimensional features of the face region in the depth image may be used to determine whether the target to be detected is a three-dimensional target or a two-dimensional target.

[0062] Figure 4 The flowchart of the image detection method according to another embodiment of the present disclosure is schematically shown.

[0063] like Figure 4 As shown, in another embodiment of the present disclosure, the image detection method is described using visible light modality and near-infrared modality as examples of preset modalities, and face recognition as a specific application scenario. It should be noted that the preset modalities that can be used in other embodiments of the present disclosure are not limited to visible light modality and near-infrared modality, the types of preset modalities used can be more than two, and the specific application scenarios are not limited to face recognition scenarios.

[0064] According to an embodiment of the present disclosure, the depth image 411 , the visible light image 421 , and the near-infrared image 431 may be acquired by photographing the target 401 to be detected using a multimodal camera.

[0065] According to an embodiment of the present disclosure, the image detection method may include two processes: a process of determining whether the target 401 to be detected is a three-dimensional target, and a detection process after determining that the target 401 to be detected is a three-dimensional target.

[0066] In the process of determining whether the target 401 to be detected is a three-dimensional target, the depth image 411 can be input into the depth model 412 to output a depth detection result 413 indicating whether the target 401 to be detected is a three-dimensional target or a two-dimensional target.

[0067] According to an embodiment of the present disclosure, the deep model 412 may be a fully trained classification model, which may use any existing model framework such as VGG Net, ResNet, ResNeXt, SE-Net, etc.

[0068] According to an embodiment of the present disclosure, when the depth detection result 413 shows that the target to be detected 401 is a two-dimensional target, a result indicating that the target to be detected 401 is an attack 442 can be output, and subsequent detection of the target to be detected 401 can be terminated.

[0069] According to the embodiments of the present disclosure, among attacks based on physical media, the devices for paper attacks and screen attacks have low manufacturing costs and are simple to operate, and account for a large proportion of all attack types. Through the detection process of the depth image 411 of the target 401 to be detected, a large proportion of attacks can be intercepted, which can effectively reduce the resource consumption of image detection and improve detection efficiency.

[0070] According to an embodiment of the present disclosure, when the depth detection result 413 shows that the target 401 to be detected is a three-dimensional target, the visible light image 421 and the near infrared image 431 of the target 401 to be detected are used to perform subsequent detection on the target 401 to be detected.

[0071] According to an embodiment of the present disclosure, before detecting the visible light image 421 and the near-infrared image 431, the visible light image 421 and / or the near-infrared image 431 may be preprocessed. The preprocessing step may include: determining the position of the target 401 to be detected in the visible light image 421 and / or the near-infrared image 431, and cropping the visible light image 421 and / or the near-infrared image 431 to a preset size with the position as the center, so as to facilitate input into the detection model.

[0072] According to embodiments of the present disclosure, noise interference can be effectively reduced by cropping visible light image 421 and / or near-infrared image 431. For example, there may be multiple targets 401 to be detected in front of a multimodal camera. By cropping the acquired images, interference from other targets on the detection results can be avoided, thereby improving the robustness of the detection results.

[0073] According to an embodiment of the present disclosure, after the visible light image 421 is input into the visible light model 422 , a visible light detection result 423 can be output; after the near infrared image 431 is input into the near infrared model 432 , a near infrared detection result 433 can be output.

[0074] According to an embodiment of the present disclosure, the visible light result 423 and the near infrared result 433 are fused according to a preset strategy to obtain a target detection result 441 that characterizes the target to be detected 401 as an attack 442 or a real target 443.

[0075] Specifically, when both visible light result 423 and near infrared result 433 indicate that target 401 to be detected is attack 442 or real target 443 , target detection result 441 can be determined to be attack 442 or real target 443 . When the detection results indicated by the visible light result 423 and the near-infrared result 433 are inconsistent, taking the example that the visible light result 423 indicates that the target to be detected 401 is the attack 442 and the near-infrared result 433 indicates that the target to be detected 401 is the real target 443, it can be determined from the initial classification model used when training the visible light model 422 that the category label value representing the attack 442 in the initial classification model is m1, and the classification threshold value can be determined from the visible light model 422 as n1, and the detection value corresponding to the detected visible light result 423 is l1; from the initial classification model used when training the near-infrared model 432, it can be determined that the category label value representing the real target 443 in the initial classification model is m2, and the classification threshold value can be determined from the near-infrared model 432 as n2, and the detection value corresponding to the detected near-infrared result 433 is l2. Afterwards, the classification weights r1 and r2 of the visible light result 423 and the near-infrared result 433 can be calculated respectively according to formula (1):

[0076]

[0077] Where l represents the detection value, n represents the classification threshold, and m represents the class label value. After obtaining the classification weights r1 and r2, r1 and r2 can be compared. If r1 < r2, target detection result 441 can be determined to be the same as near-infrared result 433 output by near-infrared model 432, indicating that target detection result 441 is a result that indicates that target 401 is a real target 443. If r1 > r2, target detection result 441 can be determined to be the same as visible light result 423 output by visible light model 422, indicating that target detection result 441 is a result that indicates that target 401 is an attack 442.

[0078] Alternatively, deep learning technology can be used to input the detection values ​​corresponding to the visible light result 423 and the near-infrared result 433 into a fully trained classification model, and the output classification result is the target detection result 441. The classification model can use any existing model framework such as VGG Net, ResNet, ResNeXt, SE-Net, etc.

[0079] According to the embodiments of the present disclosure, by utilizing the texture information, color information, and reflective properties of the target 401 to be detected contained in the visible light image 421 and the near-infrared image 431, mask-type attacks and head model-type attacks in physical medium-based attacks can be intercepted, thereby effectively improving the defense capability of the face recognition system against physical medium-based attacks.

[0080] Figure 5 The block diagram schematically shows an image detection device according to an embodiment of the present disclosure.

[0081] like Figure 5 As shown, the image detection device includes a first acquisition module 510 , a first processing module 520 and a second processing module 530 .

[0082] The first acquisition module 510 acquires images of the target to be detected in multiple preset modes.

[0083] The first processing module 520 is configured to input each frame of the image in the preset modality into an image detection model corresponding to the preset modality, and output a plurality of first image detection results.

[0084] The second processing module 530 is configured to fuse the plurality of first image detection results according to a preset strategy to obtain a target detection result for characterizing whether the target to be detected is a real target or an attack.

[0085] According to the embodiments of the present disclosure, by adopting a multimodal fusion strategy, images of the target to be detected in multiple modalities are detected, and the multiple detection results are fused to obtain a detection result of whether the target to be detected is an attack or a real target. This at least partially overcomes the technical problems existing in the related art of failing to fully utilize the advantages of each modality and having poor robustness, effectively enhances the defense capability against attacks based on physical media, and improves the robustness of the detection results.

[0086] According to an embodiment of the present disclosure, the preset modality includes a depth modality, and the image detection device further includes a second acquisition module and a third processing module.

[0087] The second processing module is used to input the image in the depth modality into the depth image detection model and output a second image detection result, wherein the second image detection result is used to characterize whether the target to be detected is a three-dimensional target or a two-dimensional target.

[0088] The third processing module is used to input each frame of the image in the preset mode into the image detection model corresponding to the preset mode when it is determined that the target to be detected is a three-dimensional target, and output a plurality of first image detection results.

[0089] According to an embodiment of the present disclosure, the third processing module is further configured to output a result indicating that the target to be detected is an attack when it is determined that the target to be detected is a two-dimensional target.

[0090] According to an embodiment of the present disclosure, the image detection device further includes a preprocessing module. The preprocessing module includes a first preprocessing unit, a second preprocessing unit, and a third preprocessing unit.

[0091] The first pre-processing unit is used to determine the position of the target to be detected in each frame of the image in the preset mode.

[0092] The second pre-processing unit is used to crop the image to a preset size with the position of the target to be detected as the center.

[0093] The third preprocessing unit is used to input the cropped image into an image detection model corresponding to a preset modality, and output a first image detection result.

[0094] According to an embodiment of the present disclosure, the second processing module 530 includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit.

[0095] The first processing unit is configured to determine, when the plurality of first image detection results are consistent, that the target detection result is consistent with the first image detection result.

[0096] The second processing unit is configured to respectively determine a plurality of detection values ​​corresponding to the plurality of first image detection results when there are inconsistencies among the plurality of first image detection results.

[0097] The third processing unit is used to calculate the classification weight of each detection value respectively.

[0098] The fourth processing unit is configured to select the first image detection result corresponding to the highest classification weight among the multiple classification weights as the target detection result.

[0099] According to an embodiment of the present disclosure, the image detection model includes an image detection model trained by an initial classification model, the initial classification model has a category label value corresponding to the image detection result, and the image detection model has a preset classification threshold.

[0100] According to an embodiment of the present disclosure, the third processing unit includes a first processing sub-unit, a second processing sub-unit, a third processing sub-unit, and a fourth processing sub-unit.

[0101] The first processing subunit is configured to determine, for each first image detection result, a category label value corresponding to the first image detection result.

[0102] The second processing subunit is configured to calculate a difference between the detection value and a preset classification threshold to obtain a first difference.

[0103] The third processing subunit is configured to calculate a difference between the category label value and a preset classification threshold value to obtain a second difference.

[0104] The fourth processing subunit is configured to obtain an absolute value of the ratio of the first difference to the second difference to obtain a classification weight.

[0105] According to an embodiment of the present disclosure, the second processing module 530 includes a fifth processing unit and a sixth processing unit.

[0106] The fifth processing unit is configured to respectively determine a plurality of detection values ​​corresponding to the plurality of first image detection results.

[0107] The sixth processing unit is used to input multiple detection values ​​into the classification model and output the target detection result.

[0108] According to the modules, submodules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of several thereof. Alternatively, according to the modules, submodules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0109] For example, any multiple of the first acquisition module 510, the first processing module 520, and the second processing module 530 can be combined into one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 510, the first processing module 520, and the second processing module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 510 , the first processing module 520 , and the second processing module 530 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0110] It should be noted that the image detection device part in the embodiment of the present disclosure corresponds to the image detection method part in the embodiment of the present disclosure. The description of the image detection device part specifically refers to the image detection method part and will not be repeated here.

[0111] Figure 6 A block diagram of an electronic device suitable for implementing an image detection method according to an embodiment of the present disclosure is schematically shown. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0112] like Figure 6 As shown, the computer electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for executing different actions of the method flow according to an embodiment of the present disclosure.

[0113] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0114] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage portion 608 including a hard disk; and a communication portion 609 including a network interface card such as a LAN card or a modem. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 610 as needed, so that a computer program read therefrom can be installed into the storage portion 608 as needed.

[0115] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0116] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0117] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0118] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603 .

[0119] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the image detection method provided by the embodiment of the present disclosure.

[0120] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0121] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0122] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0125] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image detection method, comprising: Acquire images of the target to be detected in multiple preset modes; Input each frame of the image in the preset mode into the image detection model corresponding to the preset mode, and output a plurality of first image detection results. Wherein, the preset mode includes depth mode; Inputting the image in the depth modality into a depth image detection model, and outputting a second image detection result, wherein the second image detection result is used to characterize whether the target to be detected is a three-dimensional target or a two-dimensional target. When it is determined that the target to be detected is a three-dimensional target, inputting each frame of the image in the preset modality into the image detection model corresponding to the preset modality, and outputting a plurality of first image detection results, wherein the image in the depth modality includes a plurality of regions of different depths, which are used to characterize the three-dimensional information of the target to be detected; The plurality of first image detection results are fused according to a preset strategy to obtain a target detection result for characterizing whether the target to be detected is a real target or an attack.

2. The method according to claim 1, further comprising: In the case where it is determined that the target to be detected is a two-dimensional target, a result indicating that the target to be detected is an attack is output.

3. The method according to claim 1, further comprising: For each frame of the image in the preset mode, determining the position of the target to be detected in the image; Cropping the image to a preset size with the position of the target to be detected as the center; as well as The cropped image is input into an image detection model corresponding to the preset modality, and the first image detection result is output.

4. The method according to claim 1, wherein The fusing of the plurality of first image detection results according to a preset strategy to obtain a target detection result for characterizing whether the target to be detected is a real target or an attack includes: If the plurality of first image detection results are consistent, determining that the target detection result is consistent with the first image detection result; In the case where there are inconsistencies in the detection results of the plurality of first image detection results, respectively determining a plurality of detection values ​​corresponding to the plurality of first image detection results; Calculating the classification weight of each of the detection values ​​respectively; and The first image detection result corresponding to the highest classification weight among the multiple classification weights is selected as the target detection result.

5. The method according to claim 4, wherein The image detection model is obtained by training an initial classification model, the initial classification model has a category label value corresponding to the image detection result, and the image detection model has a preset classification threshold; The step of calculating the classification weight of each detection value comprises: For each first image detection result, determining a category label value corresponding to the first image detection result; Calculating a difference between the detection value and the preset classification threshold to obtain a first difference; Calculating the difference between the category label value and the preset classification threshold to obtain a second difference; and The absolute value of the ratio of the first difference to the second difference is taken to obtain the classification weight.

6. The method according to claim 1, wherein The fusing of the plurality of first image detection results according to a preset strategy to obtain a target detection result for characterizing whether the target to be detected is a real target or an attack includes: respectively determining a plurality of detection values ​​corresponding to the plurality of first image detection results; The plurality of detection values ​​are input into a classification model, and the target detection result is output.

7. The method according to claim 1, wherein The plurality of preset modalities include a visible light modality, a near infrared modality, and a depth modality; The images under the plurality of preset modes are used to characterize the texture information, color information and reflective properties of the target to be detected.

8. An image detection device, comprising: a first acquisition module, configured to, when determining that the target to be detected is a three-dimensional target, respectively acquire images of the target to be detected in a plurality of preset modalities, wherein the preset modalities include a depth modality; a first processing module, configured to input each frame of the image in the preset modality into an image detection model corresponding to the preset modality, and output a plurality of first image detection results, wherein the image in the depth modality is input into a depth image detection model, and output a second image detection result, the second image detection result being used to characterize whether the target to be detected is a three-dimensional target or a two-dimensional target; and when the target to be detected is determined to be a three-dimensional target, input each frame of the image in the preset modality into the image detection model corresponding to the preset modality, and output a plurality of first image detection results, wherein the image in the depth modality includes a plurality of regions of different depths, which are used to characterize the three-dimensional information of the target to be detected; The second processing module is used to fuse the multiple first image detection results according to a preset strategy to obtain a target detection result used to characterize whether the target to be detected is a real target or an attack.

9. An electronic device comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7. 10 . A computer-readable storage medium having executable instructions stored thereon, wherein when the executable instructions are executed by a processor, the processor is enabled to implement the method according to claim 1 .

11. A computer program product comprising computer executable instructions, wherein the computer executable instructions are used to implement the method according to any one of claims 1 to 7 when executed.

Images