Image detection method and device, electronic equipment and storage medium

Abstract

The invention provides an image detection method and device, electronic equipment and a storage medium. The image detection method comprises the steps of determining whether a target object exists in a to-be-detected image; in response to existence of the target object in the to-be-detected image, extracting structure key point information of the target object in the to-be-detected image; and determining the classification of the target object in the to-be-detected image based on the structure key point information and the to-be-detected image. The target object is classified on the basis of the to-be-detected image and the structural key point information of the target object extracted from the to-be-detected image, so that the classification model can pay more attention to the target object in the training and discrimination process, the model can be prevented from being over-fitted to other features, the robustness of the model to data annotation noise is enhanced, and the robustness of the model to the data annotation noise is improved. And the target object classification accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image detection method, device, electronic device and storage medium. Background Art

[0002] Currently, defects in virtual portraits in AI-generated images are a common problem, such as abnormal number of fingers, abnormal limb proportions, unnatural limb joint connections, etc., which seriously affect the quality of AI-generated images. In this regard, there is currently a lack of an effective method to determine whether there are defects in virtual portraits in AI-generated images. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an image detection method, comprising:

[0005] Determine whether there is a target object in the image to be tested;

[0006] In response to the target object existing in the image to be tested, extracting structural key point information of the target object in the image to be tested;

[0007] Based on the structural key point information and the image to be tested, the classification of the target object in the image to be tested is determined.

[0008] In a second aspect, according to one or more embodiments of the present disclosure, there is provided an image detection device, comprising:

[0009] A determination unit, used to determine whether there is a target object in the image to be tested;

[0010] an extraction unit, configured to extract structural key point information of the target object in the image to be tested in response to the target object existing in the image to be tested;

[0011] A classification unit is used to determine the classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

[0012] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one memory and at least one processor; wherein the memory is used to store program code, and the processor is used to call the program code stored in the memory so that the electronic device executes the method provided according to one or more embodiments of the present disclosure.

[0013] In a fourth aspect, according to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, wherein the non-transitory computer storage medium stores a program code, and when the program code is executed by a computer device, the computer device executes the method provided according to one or more embodiments of the present disclosure.

[0014] According to one or more embodiments of the present disclosure, by classifying the target object based on the image to be tested and the structural key point information of the target object extracted from the image to be tested, the classification model can pay more attention to the target object during the training and discrimination process, and can avoid overfitting of the model to other features, thereby enhancing the robustness of the model to data annotation noise and improving the accuracy of target object classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0016] Figure 1 A flowchart of an image detection method provided by an embodiment of the present disclosure;

[0017] Figure 2 A flowchart of an image detection method provided by another embodiment of the present disclosure;

[0018] Figure 3 A flowchart of an image detection method provided by another embodiment of the present disclosure;

[0019] Figure 4 A schematic diagram of the structure of an image detection device provided by an embodiment of the present disclosure;

[0020] Figure 5 The figure is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be understood that the steps described in the embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0023] As used herein, the term "including" and its variations are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The term "responsive to" and related terms refer to a signal or event being affected to a certain extent by another signal or event, but not necessarily completely or directly. If event x occurs "responsive to" event y, x may be directly or indirectly responsive to y. For example, the occurrence of y may ultimately lead to the occurrence of x, but there may be other intermediate events and / or conditions. In other cases, y may not necessarily lead to the occurrence of x, and x may occur even if y has not yet occurred. In addition, the term "responsive to" may also mean "at least partially responsive to".

[0024] The term "determine" broadly covers a variety of actions, which may include obtaining, calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, and similar actions, and may also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and similar actions, as well as parsing, selecting, choosing, establishing, and similar actions, etc. The relevant definitions of other terms will be given in the following description. The relevant definitions of other terms will be given in the following description.

[0025] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the provisions of relevant laws and regulations.

[0026] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, and usage scenarios of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. For example, in response to receiving an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can autonomously choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the images generated by the methods provided in the embodiments of the present disclosure should be processed in accordance with the provisions of relevant laws and regulations. For example, technical measures should be taken in accordance with regulations to add a logo that does not affect user use, or a prominent logo should be placed in a reasonable location or area in accordance with regulations to inform the public of the depth synthesis situation.

[0029] It is understandable that the above-mentioned notification and process of obtaining user authorization, as well as the processing of images, are merely illustrative and do not constitute a limitation on the implementation method of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation method of the present disclosure.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] For the purposes of this disclosure, the phrase "A and / or B" means (A), (B), or (A and B).

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] There is currently a lack of targeted solutions for the identification of limb defects in virtual portraits in AI-generated content. Related technologies generally regard it as a classification problem, that is, the classification model is trained through sample images and the corresponding manually annotated labels indicating whether the limbs have defects, and the classification model is used to identify the images. However, since it is difficult to identify limb defects, the normal states of limbs under different viewing angles and different human body movements are different, which requires the classification model to have a deep understanding of the state of the limbs. The solution provided by the related technology to train the model only based on samples and manually annotated labels can easily lead to the classification model being overfitted to other easy-to-fit features in the image (such as the overall complexity of the picture) during the training process, so that the trained classification model does not truly understand limb defects. In addition, since there are inevitably mislabeling when manually labeling sample data, the existence of this data noise further increases the probability of the model fitting to other features during the training process.

[0035] In response to the above problems, the present disclosure provides an image detection method, which classifies the target object based on the image to be tested and the structural key point information of the target object extracted from the image to be tested, so that the classification model can pay more attention to the target object during the training and discrimination process, and can avoid overfitting of the model to other features, thereby enhancing the robustness of the model to data annotation noise and improving the accuracy of target object classification.

[0036] refer to Figure 1 , Figure 1 A flowchart of an image detection method 100 provided in an embodiment of the present disclosure is shown. The method 100 includes steps S110 to S130.

[0037] Step S110: Determine whether there is a target object in the image to be tested;

[0038] Step S120: In response to the target object existing in the image to be tested, extracting structural key point information of the target object in the image to be tested;

[0039] Step S130: Determine the classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

[0040] The structural key points of the target object are used to determine the structural form of the target object. In a specific embodiment, the target object may include a virtual human portrait, which may include, for example, a human portrait included in artificial intelligence generated content; accordingly, the structural key point information of the target object may include virtual human key point information, for example, may include information of some or all key points of the nose, eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles.

[0041] In some embodiments, step S130 may be used to determine whether there is a virtual human portrait defect in the image to be tested. The following is an exemplary description of a classification scenario for determining whether there is a virtual human portrait defect.

[0042] refer to Figure 2 , which shows a method 200 for determining whether a virtual portrait in an image has defects, including steps S201 to S204.

[0043] In step 201, an image to be tested is obtained. Exemplarily, the image to be tested may include artificial intelligence generated content or other virtual images.

[0044] In step 202, human body detection is performed on the image to be tested to determine whether there is a virtual human portrait. In some embodiments, whether there is a virtual human portrait in the image to be tested can be determined by a target detection algorithm. The task of target detection is to find all targets (objects) of interest in the image and determine their categories and positions. The target detection algorithm can adopt a target detection algorithm such as a region-based convolutional neural network, a single-step multi-frame target detection algorithm, a retinal network-based target detection algorithm, etc., but the present disclosure is not limited thereto.

[0045] If it is determined that there is no virtual human portrait, it can be directly determined that there is no virtual human portrait defect in the image to be tested. If it is determined that there is a virtual human portrait, step 203 is executed.

[0046] In step 203, the virtual human key point information of the image to be tested is extracted. In some embodiments, the virtual human key point information of the image to be tested can be extracted by a human key point estimation algorithm, and the human key point estimation algorithm can adopt a human posture estimation method such as a high-resolution network, a convolutional posture machine, a stacked hourglass network, and a cascaded pyramid network, but the present disclosure is not limited thereto. In a specific embodiment, the eight coordinates corresponding to the hands, elbows, feet, and legs of the virtual human figure in the image to be tested can be extracted as virtual human key point information.

[0047] In step 204, the virtual human key point information and the image to be tested can be used together as inputs of the trained classification model to determine whether there are defects in the virtual human portrait in the image to be tested. In a specific embodiment, the classification model is trained to be able to determine the confidence level of the existence of defects in the virtual human portrait in the image based on the input image and the virtual human key point information. Exemplarily, the sample image and the virtual human key point information extracted based on the sample image can be used as the input of the model, and the manually annotated label corresponding to the sample image (for example, whether there are defects in the virtual human portrait in the sample image) can be used as the expected output to train the classification model.

[0048] The manually annotated label can be related to whether the sample image has a virtual portrait defect, and can also refer to other classification types according to other application needs, which is not limited in the present disclosure. In a specific implementation, the positive label can be 1 and the negative label can be 0.

[0049] It should be noted that the image detection method provided by the present disclosure can be used not only to classify scenarios for determining whether a virtual human portrait has defects, but can also be used to classify target objects with specific structures such as animals, robots, and virtual images, and the present disclosure does not impose any limitations thereon.

[0050] According to one or more embodiments of the present disclosure, by classifying the target object based on the image to be tested and the structural key point information of the target object extracted from the image to be tested, the classification model can pay more attention to the target object during the training and discrimination process, and can avoid overfitting of the model to other features, thereby enhancing the robustness of the model to data annotation noise and improving the accuracy of target object classification.

[0051] In some embodiments, step S130 includes:

[0052] Step A1: generating a first target feature based on the image to be tested, wherein the first target feature includes a feature associated with the target object;

[0053] Step A2: performing feature conversion on the structural key point information to obtain a second target feature;

[0054] Step A3: generating a third target feature based on the first target feature and the second target feature;

[0055] Step A4: Determine the classification of the target object in the image to be tested based on the third target feature.

[0056] In some embodiments, the number of channels corresponding to the second target feature is the same as the number of channels corresponding to the first target feature. By making the number of channels corresponding to the second target feature the same as the number of channels corresponding to the first target feature, it is convenient to subsequently connect the first target feature and the second target feature to obtain the third target feature.

[0057] In some embodiments, before determining the classification of the target object in the image to be tested based on the third target feature, the third target feature is subjected to semantic unit fusion processing, so that the semantic unit in the third target feature can fuse the information of other semantic units in the third target feature, that is, the semantic units can be fused with each other. A semantic unit is a unit with semantic information, such as a token. Further, the classification of the target object in the image to be tested can be determined based on the third target feature after the semantic unit fusion processing.

[0058] In this embodiment, by performing semantic unit fusion processing on the third target feature, the features from the image to be tested and the features from the structural key point information can be further fused to a certain extent at the semantic unit level, thereby further enhancing the classification model's attention to the area of ​​the target object part in the image.

[0059] In some specific embodiments, the self-attention mechanism in the conversion network can be used to determine the fusion weight based on the correlation between the semantic units to achieve semantic unit fusion, or the semantic unit fusion can be achieved by expanding the size of the convolution kernel in the convolutional neural network, or by fully connecting between tokens in the multi-layer perceptron, but the present disclosure is not limited to this.

[0060] It should be noted that the semantic unit fusion processing in this embodiment can be a global semantic unit fusion or a fusion of some semantic units, and the present disclosure does not limit this.

[0061] In some embodiments, step A4 includes:

[0062] Step a41: inputting the third target feature into a classification network to obtain a confidence level that a defect exists in the target object and / or a confidence level that a defect does not exist in the target object;

[0063] Step a42: based on the confidence level that the target object has defects and / or the confidence level that the target object does not have defects, obtain the category of the target object.

[0064] In some embodiments, before the third target feature is input into the classification network, other processing may be performed on the third target feature, such as semantic unit fusion processing, preset dimension averaging processing, etc., which is not limited in the present disclosure.

[0065] See also Figure 3 , the following is an exemplary description of a classification scenario for determining whether a virtual portrait has defects.

[0066] In the first step, a normalized tensor Ti can be obtained based on the virtual human key point information of the image to be tested, which can be expressed as (B, N, C), where B represents the batch size of the training sample, that is, the number of data (samples) passed to the program for training at a single time; N represents the number of semantic units; C represents the number of channels. For example, assuming that the coordinates (x, y) of 8 human key points in the x and y directions are obtained for each image to be tested, then N can be 8 and C can be 2.

[0067] In the second step, the tensor Ti is input into the feature extraction network for feature conversion to obtain the tensor Tf, which can be expressed as (B, N1, C1). In a specific embodiment, the feature extraction network can be a conversion network, which is a neural network model based on the self-attention mechanism for processing sequence data.

[0068] In the third step, the image to be tested (such as an RGB image) is input into the backbone network to extract features, and a tensor Tm is obtained, which can be expressed as (B, N2, C1). A well-trained backbone network is expected to be used to extract features associated with the classification task from the image to be tested. In the classification scenario of the virtual portrait, it is used to extract features about the structure and quantity of the virtual portrait. Of course, in practical applications, the features extracted by the backbone network may also include features that are not expected to be extracted, such as features of buildings or other objects in the image that are not related to the virtual portrait. In a specific embodiment, the backbone network can use a visual transformation network or other image feature extraction network, but the present disclosure is not limited to this.

[0069] In the fourth step, the tensor Tf and the tensor Tm are connected in the dimension of the semantic unit to obtain a tensor Tc, which can be expressed as (B, N1 + N2, C1). Exemplarily, the connection operation can be implemented by a connection function (Concat), which can connect features in a specified dimension.

[0070] In the fifth step, the tensor Tc is input into the feature fusion network to obtain the tensor To, which can be expressed as (B, N3, C2). The feature fusion network can be used to perform semantic unit fusion processing on the input features, so that each semantic unit in the feature can be fused based on its correlation with other semantic units, thereby improving the degree of information interaction. Exemplarily, the feature fusion network can be a conversion network.

[0071] In the sixth step, the tensor To is averaged in the dimension of the semantic unit (i.e., "N" in (B, N, C)), and is input into the classification head network to obtain the final classification of whether the virtual portrait has defects. The classification result can be expressed as the confidence of "the virtual portrait has defects" and / or the confidence of "the virtual portrait does not have defects". Exemplarily, the classification head network can be a fully connected layer. In this step, by performing the average operation on the tensor To in the dimension of the semantic unit, it can be easily processed by the classification head network.

[0072] In this embodiment, the classification network includes a feature extraction network, a backbone network, a feature fusion network and a classification head network. Exemplarily, when the classification network is trained, the sample image and its corresponding virtual human key point information can be input into the backbone network and the feature extraction network respectively, and the prediction results are obtained by passing through the feature fusion network and the classification head network in turn, and the parameters of the feature extraction network, the backbone network, the feature fusion network and the classification head network are adjusted based on the loss of the label corresponding to the prediction result and the sample image (e.g., cross entropy loss) to converge the loss function. In a specific embodiment, the backbone network can be initialized by pre-training parameters to accelerate convergence.

[0073] Accordingly, refer to Figure 4 According to an embodiment of the present disclosure, an image detection device 400 is provided, comprising:

[0074] A determination unit 401 is used to determine whether there is a target object in the image to be tested;

[0075] An extraction unit 402 is used for extracting structural key point information of the target object in the image to be tested in response to the target object existing in the image to be tested;

[0076] The classification unit 403 is used to determine the classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

[0077] In some embodiments, the target object includes a virtual human portrait, and the structural key point information of the target object includes virtual human body key point information.

[0078] In some embodiments, the classification unit comprises:

[0079] A feature extraction subunit, configured to generate a first target feature based on the image to be measured, wherein the first target feature includes a feature associated with the target object;

[0080] A feature conversion subunit, used for performing feature conversion on the structural key point information to obtain a second target feature;

[0081] a feature generating subunit, configured to generate a third target feature based on the first target feature and the second target feature; and

[0082] A classification subunit is used to determine the classification of the target object in the image to be tested based on the third target feature.

[0083] In some embodiments, the classification subunit is used to input the third target feature into a classification network to obtain a confidence level that a defect exists in the target object and / or a confidence level that a defect does not exist in the target object, and based on the confidence level that a defect exists in the target object and / or the confidence level that a defect does not exist in the target object, obtain a category of the target object.

[0084] In some embodiments, the image detection device further comprises:

[0085] A feature fusion subunit, used for performing semantic unit fusion processing on the third target feature before determining the classification of the target object in the image to be tested based on the third target feature, so that the semantic unit in the third target feature can fuse the information of other semantic units in the third target feature;

[0086] The classification subunit is used to determine the classification of the target object in the image to be tested based on the third target feature after the semantic unit fusion processing.

[0087] In some embodiments, the determining unit is further configured to determine that the image to be tested does not have a target object defect in response to the target object not being present in the image to be tested.

[0088] For the embodiments of the device, since they basically correspond to the method embodiments, the relevant parts refer to the partial description of the method embodiments. The device embodiments described above are merely illustrative, and the modules described as separation modules may or may not be separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present embodiment. Those of ordinary skill in the art can understand and implement without paying creative work.

[0089] Accordingly, according to one or more embodiments of the present disclosure, there is provided an electronic device, including:

[0090] at least one memory and at least one processor;

[0091] The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to enable the electronic device to execute the image detection method provided according to one or more embodiments of the present disclosure.

[0092] Accordingly, according to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, which stores program code, and the program code can be executed by a computer device to enable the computer device to perform the image detection method provided according to one or more embodiments of the present disclosure.

[0093] Reference below Figure 5, which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 800 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0094] like Figure 5 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0095] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0096] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0097] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0099] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0100] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method of the present disclosure.

[0101] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0104] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0105] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] According to one or more embodiments of the present disclosure, there is provided an image detection method, comprising: determining whether a target object exists in an image to be tested; in response to the presence of the target object in the image to be tested, extracting structural key point information of the target object in the image to be tested; and determining a classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

[0107] According to one or more embodiments of the present disclosure, the target object includes a virtual human portrait, and the structural key point information of the target object includes virtual human key point information.

[0108] According to one or more embodiments of the present disclosure, based on the structural key point information and the image to be tested, determining the classification of the target object in the image to be tested includes: generating a first target feature based on the image to be tested, the first target feature including a feature associated with the target object; performing feature conversion on the structural key point information to obtain a second target feature; generating a third target feature based on the first target feature and the second target feature; and determining the classification of the target object in the image to be tested based on the third target feature.

[0109] According to one or more embodiments of the present disclosure, determining the classification of the target object in the image to be tested based on the third target feature includes: inputting the third target feature into a classification network to obtain a confidence level that a defect exists on the target object and / or a confidence level that a defect does not exist on the target object; and obtaining a category of the target object based on the confidence level that a defect exists on the target object and / or the confidence level that a defect does not exist on the target object.

[0110] According to one or more embodiments of the present disclosure, the method further includes: before determining the classification of the target object in the image to be tested based on the third target feature, performing semantic unit fusion processing on the third target feature so that the semantic unit in the third target feature can fuse information of other semantic units in the third target feature; determining the classification of the target object in the image to be tested based on the third target feature includes: determining the classification of the target object in the image to be tested based on the third target feature after the semantic unit fusion processing.

[0111] The method provided according to one or more embodiments of the present disclosure further includes: in response to the target object not existing in the image to be tested, determining that the image to be tested does not have a target object defect.

[0112] According to one or more embodiments of the present disclosure, there is provided an image detection device, comprising: a determination unit, for determining whether a target object exists in an image to be tested; an extraction unit, for extracting structural key point information of the target object in the image to be tested in response to the presence of the target object in the image to be tested; and a classification unit, for determining a classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

[0113] According to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one memory and at least one processor; wherein the memory is used to store program code, and the processor is used to call the program code stored in the memory so that the electronic device executes the image detection method provided according to one or more embodiments of the present disclosure.

[0114] According to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, wherein the non-transitory computer storage medium stores a program code, and when the program code is executed by a computer device, the computer device executes the image detection method provided according to one or more embodiments of the present disclosure.

[0115] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0116] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0117] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image detection method, characterized in that: include: Determine whether there is a target object in the image to be tested; In response to the target object existing in the image to be tested, extracting structural key point information of the target object in the image to be tested; Based on the structural key point information and the image to be tested, the classification of the target object in the image to be tested is determined.

2. The method according to claim 1, characterized in that The target object includes a virtual human portrait, and the structural key point information of the target object includes virtual human key point information.

3. The method according to claim 1, characterized in that Determining the classification of the target object in the image to be tested based on the structural key point information and the image to be tested includes: generating a first target feature based on the image to be measured, wherein the first target feature includes a feature associated with the target object; Performing feature conversion on the structural key point information to obtain a second target feature; generating a third target feature based on the first target feature and the second target feature; The classification of the target object in the image to be tested is determined based on the third target feature.

4. The method according to claim 3, characterized in that The determining the classification of the target object in the image to be tested based on the third target feature includes: Inputting the third target feature into a classification network to obtain a confidence level that a defect exists in the target object and / or a confidence level that a defect does not exist in the target object; Based on the confidence level that a defect exists on the target object and / or the confidence level that a defect does not exist on the target object, a category of the target object is obtained.

5. The method according to claim 3, characterized in that: The method further comprises: before determining the classification of the target object in the image to be tested based on the third target feature, performing semantic unit fusion processing on the third target feature so that the semantic unit in the third target feature can fuse information of other semantic units in the third target feature; The determining the classification of the target object in the image to be tested based on the third target feature includes: determining the classification of the target object in the image to be tested based on the third target feature after semantic unit fusion processing.

6. The method according to claim 1, characterized in that Also includes: In response to the target object not existing in the image to be tested, it is determined that the image to be tested does not have a target object defect.

7. An image detection device, characterized in that: include: A determination unit, used to determine whether a target object exists in the image to be tested; an extraction unit, configured to extract structural key point information of the target object in the image to be tested in response to the target object existing in the image to be tested; A classification unit is used to determine the classification of the target object in the image to be tested based on the structural key point information and the image to be tested.

8. An electronic device, characterized in that: include: at least one memory and at least one processor; The memory is used to store program codes, and the processor is used to call the program codes stored in the memory to enable the electronic device to execute the method according to any one of claims 1 to 6.

9. A non-transitory computer storage medium, characterized in that The non-transitory computer storage medium stores a program code, and when the program code is executed by a computer device, the computer device executes the method according to any one of claims 1 to 6.

Images