Method, device, equipment and storage medium for constructing image recognition model
Through the joint training and cascade combination of the initial super-resolution model and recognition model, the problem of poor face image quality in unconstrained natural scenes is solved, and the accuracy and robustness of face recognition are improved.
Patent Information
- Application Number
- CN202110856547.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-28
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure CN113591675B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, specifically the field of computer vision and deep learning technology, and in particular to methods, devices, equipment and storage media for constructing image recognition models, which can be applied to scenarios such as face recognition. Background Art
[0002] Face recognition is one of the earliest and most widely used technologies in computer vision, especially in the fields of security and mobile payment. With the widespread application of deep learning in face recognition technology, the accuracy of face recognition based on deep learning has been greatly improved.
[0003] However, in more general unconstrained natural scenes, after the camera captures the video stream, the captured face images often appear blurry, with small face areas and other poor quality problems, resulting in a low recognition pass rate or a high misrecognition rate. Summary of the invention
[0004] The present invention provides a method, device, equipment and storage medium for constructing an image recognition model.
[0005] According to a first aspect of the present disclosure, a method for constructing an image recognition model is provided, comprising: obtaining an input image set; jointly training an initial super-resolution model and an initial recognition model using the input image set to obtain a trained super-resolution model and a recognition model; and combining the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model.
[0006] According to the second aspect of the present disclosure, there is provided an image recognition method, comprising: acquiring an image to be recognized; inputting the image to be recognized into an image recognition model, and outputting a recognition result corresponding to the image to be recognized, wherein the image recognition model is obtained by a method as described in any implementation method in the first aspect.
[0007] According to a third aspect of the present disclosure, there is provided a device for constructing an image recognition model, comprising: a first acquisition module, configured to acquire an input image set; a training module, configured to jointly train an initial super-resolution model and an initial recognition model using the input image set to obtain a trained super-resolution model and a recognition model; and a combination module, configured to combine the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model.
[0008] According to the fourth aspect of the present disclosure, an image recognition device is provided, comprising: a second acquisition module, configured to acquire an image to be recognized; an output module, configured to input the image to be recognized into an image recognition model, and output a recognition result corresponding to the image to be recognized, wherein the image recognition model is obtained by a method as described in any implementation method in the first aspect.
[0009] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner of the first aspect or the second aspect.
[0010] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation manner of the first aspect or the second aspect.
[0011] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the method described in any implementation manner of the first aspect or the second aspect.
[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0014] Figure 1 is an exemplary system architecture diagram in which the present disclosure may be applied;
[0015] Figure 2 is a flowchart of an embodiment of a method for constructing an image recognition model according to the present disclosure;
[0016] Figure 3 is a schematic diagram of an application scenario of the method for constructing an image recognition model according to the present disclosure;
[0017] Figure 4 is a flowchart of another embodiment of a method for constructing an image recognition model according to the present disclosure;
[0018] Figure 5 is a flowchart of another embodiment of a method for constructing an image recognition model according to the present disclosure;
[0019] Figure 6 is a flow chart of an embodiment of an image recognition method according to the present disclosure;
[0020] Figure 7 is a structural schematic diagram of an embodiment of a device for constructing an image recognition model according to the present disclosure;
[0021] Figure 8 is a structural schematic diagram of an embodiment of an image recognition device according to the present disclosure;
[0022] Fig. 9 It is a block diagram of an electronic device used to implement the method for constructing an image recognition model according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0025] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the method for constructing an image recognition model or an apparatus for constructing an image recognition model disclosed in the present invention can be applied.
[0026] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0027] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send information, etc. Various client applications can be installed on the terminal devices 101, 102, 103.
[0028] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above electronic devices. They may be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.
[0029] The server 105 can provide various services. For example, the server 105 can analyze and process the input image set obtained from the terminal devices 101, 102, and 103, and generate a processing result (eg, an image recognition model).
[0030] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.
[0031] It should be noted that the method for constructing the image recognition model provided in the embodiments of the present disclosure is generally executed by the server 105 , and accordingly, the device for constructing the image recognition model is generally disposed in the server 105 .
[0032] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0033] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a method for constructing an image recognition model according to the present disclosure. The method for constructing an image recognition model includes the following steps:
[0034] Step 201, obtaining an input image set.
[0035] In this embodiment, the execution subject of the method for constructing the image recognition model (for example Figure 1 The server 105 shown in the figure can obtain an input image set, wherein the input image set can include at least one input image.
[0036] It should be noted that the input images in the input image set may be multiple images containing human faces that are pre-collected in various ways. For example, the input image set may be multiple images obtained from an existing image library. For another example, the input image set may also be multiple images collected in real time by an image sensor (such as a camera sensor) in an actual application scenario. The present disclosure does not make any specific limitations on this.
[0037] Step 202 , jointly train the initial super-resolution model and the initial recognition model using the input image set to obtain trained super-resolution model and recognition model.
[0038] In this embodiment, the execution subject may use the input image set obtained in step 201 to jointly train the initial super-resolution model and the initial recognition model, thereby obtaining a trained super-resolution model and recognition model.
[0039] Here, the initial super-resolution model and the initial recognition model can be predetermined. For example, the initial super-resolution model can be SRCNN (Super-Resolution Convolutional Neural Networks), FSRCNN (Fast Super-Resolution Convolutional Neural Networks), SRGAN (Super-Resolution Generative Adversarial Network) and other models; and the initial recognition model can be an existing classification and recognition model such as the ResNet (Residual Network) series, or a model designed according to actual needs.
[0040] The execution subject can use the input image set obtained in step 201 to jointly train the initial super-resolution model and the initial recognition model, so as to adjust the parameters of the initial super-resolution model and the initial recognition model through the input image set, and stop the training when the joint training stop condition is met, so as to obtain the trained super-resolution model and recognition model. The joint training stop condition may include the preset number of training times, or the value of the loss function no longer decreases, or a certain accuracy threshold is set, and the training is stopped when the preset threshold is reached.
[0041] Step 203, combining the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model.
[0042] In this embodiment, the execution subject can combine the trained super-resolution model and the recognition model obtained in step 202 in a cascade manner to obtain an image recognition model. This step sets the trained super-resolution model before the recognition model, which can add more information to the recognition model, thereby obtaining a better effect.
[0043] The method for constructing an image recognition model provided in the embodiment of the present disclosure first obtains an input image set; then uses the input image set to jointly train an initial super-resolution model and an initial recognition model to obtain a trained super-resolution model and a recognition model; finally, the trained super-resolution model and the recognition model are combined in a cascade manner to obtain an image recognition model. The method for constructing an image recognition model in this embodiment, by jointly training an initial super-resolution model and an initial recognition model, alleviates the impact of images of different resolutions on the classification task, improves the robustness of the image recognition model to low-quality data, and thereby improves the recognition accuracy of the image recognition model.
[0044] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0045] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the method for constructing an image recognition model according to the present disclosure. Figure 3 In the application scenario, first, the execution subject 301 obtains the input image set 302. Then, the execution subject 301 uses the input image set 302 to jointly train the initial super-resolution model and the initial recognition model to obtain the trained super-resolution model 303 and the recognition model 304. Finally, the execution subject 301 combines the trained super-resolution model 303 and the recognition model 304 in a cascade manner to obtain the image recognition model 305.
[0046] Continue to refer Figure 4 , Figure 4 A process 400 of another embodiment of a method for constructing an image recognition model according to the present disclosure is shown. The method for constructing an image recognition model comprises the following steps:
[0047] Step 401, obtaining an input image set.
[0048] Step 401 is basically the same as step 201 of the aforementioned embodiment. The specific implementation method can refer to the aforementioned description of step 201, which will not be repeated here.
[0049] Step 402, using the input image set and the restored image set corresponding to the input image set to calculate the loss function of the initial super-resolution model, using the gradient descent method to update the parameters of the initial super-resolution model, to obtain a trained super-resolution model.
[0050] In this embodiment, the execution subject of the method for constructing the image recognition model (for example Figure 1 The server 105 shown in the figure can determine the restored image corresponding to each image in the input image set after obtaining the input image set, thereby obtaining the restored image set corresponding to the input image set.
[0051] Then, the execution entity can use the input images in the input image set and the corresponding restored images in the restored image set to calculate the loss function of the initial super-resolution model, and use the gradient descent method to iteratively solve it step by step, so as to obtain the minimized loss function and model parameter value.
[0052] Finally, these obtained model parameter values are used to update the parameters of the initial super-resolution model to obtain a trained super-resolution model, thereby improving the quality of the results.
[0053] Step 403, based on the distance between the features of the input image set and the images in the restored image set, the loss function of the initial recognition model is calculated, and the parameters of the initial recognition model are updated using a gradient descent method to obtain a trained recognition model.
[0054] In this embodiment, the above-mentioned execution entity can calculate the loss function of the initial recognition model based on the distance between the features of the images in the input image set and the restored image set. For example, the input image set and the images in the restored image set can be merged first to obtain the final image set, and then the distance between the image features in the obtained image set is calculated, and the loss function of the initial recognition model is calculated based on these distances.
[0055] Afterwards, the gradient descent method is used to iteratively solve the problem step by step to obtain the minimized loss function and model parameter values. These model parameter values are then used to update the parameters of the initial recognition model to obtain a trained recognition model, thereby improving the classification accuracy of the recognition model.
[0056] In some optional implementations of this embodiment, the above-mentioned gradient descent method is a stochastic gradient descent method. The use of the stochastic gradient descent method can more quickly obtain the minimized loss function and model parameter values, thereby improving the efficiency of model training.
[0057] Step 404, connecting the output end of the part before the loss function in the trained super-resolution model to the input end of the recognition model to obtain the image recognition model.
[0058] In this embodiment, the execution subject can connect the output end of the part before the loss function in the trained super-resolution model to the input end of the recognition model, thereby obtaining an image recognition model. By placing the trained super-resolution model before the recognition model, more information can be added to the recognition model, thereby obtaining a better effect.
[0059] from Figure 4 It can be seen that Figure 2 Compared with the corresponding embodiments, the method for constructing the image recognition model in this embodiment highlights the step of using the input image set to train the initial super-resolution model and the initial recognition model, which improves the efficiency of model training, and also improves the accuracy of the trained super-resolution model and recognition model, and has a wider range of applications.
[0060] Continue to refer Figure 5 , Figure 5 A process 500 of another embodiment of a method for constructing an image recognition model according to the present disclosure is shown. The method for constructing an image recognition model includes the following steps:
[0061] Step 501: Obtain an input image set.
[0062] Step 501 is basically the same as step 401 of the aforementioned embodiment. For the specific implementation method, reference may be made to the aforementioned description of step 401 and will not be repeated here.
[0063] Step 502: downsample the images in the input image set to obtain a downsampled image set.
[0064] In this embodiment, the execution subject of the method for constructing the image recognition model (for example Figure 1 The server 105 shown in the figure can downsample each image in the input image set to obtain a corresponding downsampled image, and then obtain a downsampled image set including the downsampled image corresponding to each input image in the input image set. The downsampled image obtained by this step is a low-quality image that is more suitable for actual application scenarios.
[0065] Step 503: Use the initial super-resolution model to restore the images in the downsampled image set to obtain a restored image set.
[0066] In this embodiment, the above-mentioned execution entity can use the initial super-resolution model to restore each downsampled image in the downsampled image set, so as to obtain a corresponding restored image, which is a high-quality image obtained by restoring the low-quality image obtained in step 502, and then obtain a restored image set including the restored images corresponding to each downsampled image in the downsampled image set.
[0067] Step 504 , based on the input image set and the restored image set, the reconstruction loss of the initial super-resolution model is calculated, and the parameters of the initial super-resolution model are updated using a gradient descent method to obtain a trained super-resolution model.
[0068] In this embodiment, the above-mentioned execution entity can use the input image in the input image set and the restored image corresponding to the input image in the restored image set to calculate the reconstruction loss, and use the gradient descent method to iteratively solve step by step to obtain the minimized loss function and model parameter values, and then use these obtained model parameter values to update the parameters of the initial super-resolution model, so as to obtain a trained super-resolution model.
[0069] The above steps improve the result quality of the super-resolution model.
[0070] Step 505: merge the input image set, the downsampled image set and the restored image set to obtain a target image set.
[0071] In this embodiment, the execution entity may merge the input image set, the downsampled image set, and the restored image set to obtain a target image set.
[0072] Step 506: extract features of images in the target image set, and calculate distances between the features of images in the target image set.
[0073] In this embodiment, the execution entity may extract features of each image in the target image set, and calculate the distances between the images in the target image set based on the extracted features.
[0074] Optionally, before obtaining the input image set, the input images in the input image set may be labeled, and an ID (Identity Document) may be assigned to each target object, where the target object is the object represented by the face in the input image. Then, the input image corresponding to each target object in the input image set should have the same ID, and the IDs of the downsampled image and the restored image correspond to the ID of the input image.
[0075] On this basis, in this step, the distance between images can be calculated based on the ID, the distance between all images with the same ID can be calculated based on the extracted image features, and then the distance between images with different IDs can be calculated.
[0076] Step 507, based on the distance calculation, a binary loss function of the initial recognition model is calculated, and the parameters of the initial recognition model are updated using a gradient descent method to obtain a trained recognition model.
[0077] In this embodiment, the execution entity may calculate the binary loss function of the initial recognition model based on the distance calculated in step 506 .
[0078] Optionally, when two images have the same ID, the loss function is the square of the distance between the two images. When two images have different IDs, the margin between the two images is calculated first, and then the max is calculated to get the loss value. That is, the distance between images with the same ID is closer, and the distance between all images with different IDs is farther, so as to increase the difference between classes and reduce the difference within classes.
[0079] Then the gradient descent method is used to iterate the solution step by step to obtain the minimized loss function and model parameter values, and then these model parameter values are used to update the parameters of the initial recognition model to obtain the trained recognition model.
[0080] The above steps improve the classification accuracy of the recognition model.
[0081] Step 508, connecting the output end of the part before the loss function in the trained super-resolution model to the input end of the recognition model to obtain the image recognition model.
[0082] Step 508 is basically the same as step 404 in the aforementioned embodiment. For the specific implementation method, reference may be made to the aforementioned description of step 404 and will not be repeated here.
[0083] from Figure 5 It can be seen that Figure 4 Compared with the corresponding embodiments, the method for constructing the image recognition model in this embodiment calculates the reconstruction loss of the initial super-resolution model and the binary loss function of the initial recognition model based on the input image set and the restored image set, and uses the gradient descent method to update the parameters of the initial super-resolution model and the initial recognition model to obtain the trained super-resolution model and recognition model, thereby improving the result quality of the super-resolution model and the classification accuracy of the recognition model.
[0084] Continue to refer Figure 6 , Figure 6 A process 600 of an embodiment of an image recognition method according to the present disclosure is shown. The image recognition method comprises the following steps:
[0085] Step 601: Obtain an image to be recognized.
[0086] In this embodiment, the execution subject of the image recognition method (for example Figure 1 The server 105 shown in the figure can obtain an image to be recognized, wherein the image to be recognized can be an image containing a face collected by a camera sensor in an actual application scenario of face recognition.
[0087] Step 602: input the image to be recognized into the image recognition model, and output the recognition result corresponding to the image to be recognized.
[0088] In this embodiment, the above-mentioned execution entity can input the image to be recognized into the image recognition model, and output the recognition result corresponding to the image to be recognized, wherein the image recognition model can be obtained through the construction method of the image recognition model in the above-mentioned embodiment.
[0089] After the above-mentioned execution subject inputs the image to be identified into the image recognition model, the image recognition model will first restore the image to be identified to obtain the corresponding restored image; then extract the features of the image to be identified and the restored image, classify them based on the features, thereby obtaining the corresponding recognition result, and output the recognition result.
[0090] The image recognition method provided by the embodiment of the present disclosure first obtains an image to be recognized; then the image to be recognized is input into an image recognition model, and a recognition result corresponding to the image to be recognized is output. The image recognition method in this embodiment recognizes the image to be recognized by using a pre-trained image recognition model, thereby improving the accuracy of the recognition result.
[0091] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for constructing an image recognition model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0092] like Figure 7 As shown, the apparatus 700 for constructing an image recognition model of this embodiment includes: a first acquisition module 701, a training module 702 and a combination module 703. The first acquisition module 701 is configured to acquire an input image set; the training module 702 is configured to jointly train the initial super-resolution model and the initial recognition model using the input image set to obtain a trained super-resolution model and a recognition model; the combination module 703 is configured to combine the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model.
[0093] In the apparatus 700 for constructing an image recognition model in this embodiment, the specific processing of the first acquisition module 701, the training module 702 and the combination module 703 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of steps 201 - 203 in the corresponding embodiment are not repeated here.
[0094] In some optional implementations of the present embodiment, the training module includes: a first updating submodule, configured to calculate the loss function of the initial super-resolution model using the input image set and the restored image set corresponding to the input image set, and to update the parameters of the initial super-resolution model using the gradient descent method; a second updating submodule, configured to calculate the loss function of the initial recognition model based on the distance between the features of the images in the input image set and the restored image set, and to update the parameters of the initial recognition model using the gradient descent method.
[0095] In some optional implementations of the present embodiment, the first updating submodule includes: a downsampling unit, configured to downsample images in the input image set to obtain a downsampled image set; a restoration unit, configured to restore images in the downsampled image set using an initial super-resolution model to obtain a restored image set; and a first calculation unit, configured to calculate the reconstruction loss of the initial super-resolution model based on the input image set and the restored image set.
[0096] In some optional implementations of the present embodiment, the second update submodule includes: a merging unit, configured to merge the input image set, the downsampled image set and the restored image set to obtain a target image set; an extraction unit, configured to extract features of images in the target image set; a second calculation unit, configured to calculate the distance between the features of the images in the target image set; and a third calculation unit, configured to calculate the binary loss function of the initial recognition model based on the distance.
[0097] In some optional implementations of this embodiment, the combination module includes: a connection submodule configured to connect the output end of the part before the loss function in the trained super-resolution model to the input end of the recognition model.
[0098] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an image recognition device, and the device embodiment is Figure 6 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0099] like Figure 8 As shown, the image recognition device 800 of this embodiment includes: a second acquisition module 801 and an output module 802. The second acquisition module 801 is configured to acquire the image to be recognized; the output module 802 is configured to input the image to be recognized into the image recognition model and output the recognition result corresponding to the image to be recognized.
[0100] In this embodiment, in the image recognition device 800, the specific processing of the second acquisition module 801 and the output module 802 and the technical effects thereof can be referred to in Figure 6The relevant descriptions of steps 601-602 in the corresponding embodiment are not repeated here.
[0101] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0102] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0103] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0104] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0105] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the construction method of the image recognition model or the image recognition method. For example, in some embodiments, the construction method of the image recognition model or the image recognition method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the construction method of the image recognition model or the image recognition method described above may be executed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the image recognition model construction method or the image recognition method in any other appropriate manner (for example, by means of firmware).
[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0107] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0108] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0110] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0111] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0113] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for constructing an image recognition model, comprising: Acquire an input image set, wherein the input images in the input image set include target objects, the target objects are objects represented by faces in the input images, and each target object is assigned an identity identification number; Using the input image set to jointly train an initial super-resolution model and an initial recognition model to obtain a trained super-resolution model and a recognition model; Combining the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model; The step of jointly training the initial super-resolution model and the initial recognition model using the input image set includes: Calculating a loss function of an initial super-resolution model using the input image set and a restored image set corresponding to the input image set, and updating parameters of the initial super-resolution model using a gradient descent method; The method comprises calculating the distance between the features of the input image set and the images in the restored image set based on the identity identification number, comprising: respectively calculating the distance between the features of images with the same identity identification number and the distance between the features of images with different identity identification numbers; calculating the loss function of the initial recognition model based on the distance between the features of the input image set and the images in the restored image set, comprising: when two images have the same identity identification number, the loss function is the square of the distance between the features of the two images; and updating the parameters of the initial recognition model using the gradient descent method.
2. The method according to claim 1, wherein: The step of calculating the loss function of the initial super-resolution model by using the input image set and the restored image set corresponding to the input image set includes: Downsampling the images in the input image set to obtain a downsampled image set; Restoring images in the downsampled image set using the initial super-resolution model to obtain a restored image set; The reconstruction loss of the initial super-resolution model is calculated based on the input image set and the restored image set.
3. The method according to claim 2, wherein: The loss function of the initial recognition model is calculated based on the distance between the features of the input image set and the images in the restored image set, comprising: Merging the input image set, the downsampled image set and the restored image set to obtain a target image set; Extracting features of images in the target image set; Calculating distances between features of images in the target image set; A bigram loss function of the initial recognition model is calculated based on the distance.
4. The method according to any one of claims 1 to 3, wherein: The gradient descent method is a stochastic gradient descent method.
5. The method according to claim 1, wherein: The step of combining the trained super-resolution model and the recognition model in a cascade manner includes: The output end of the part before the loss function in the trained super-resolution model is connected to the input end of the recognition model.
6. An image recognition method, comprising: Obtain an image to be recognized; The image to be recognized is input into an image recognition model, and a recognition result corresponding to the image to be recognized is output, wherein the image recognition model is obtained by the method for constructing an image recognition model according to any one of claims 1 to 5.
7. A device for constructing an image recognition model, comprising: A first acquisition module is configured to acquire an input image set, wherein the input images in the input image set include target objects, the target objects are objects represented by faces in the input images, and each target object is assigned an identity identification number; A training module is configured to jointly train the initial super-resolution model and the initial recognition model using the input image set to obtain a trained super-resolution model and a recognition model; A combination module is configured to combine the trained super-resolution model and the recognition model in a cascade manner to obtain an image recognition model; Wherein, the training module includes: A first updating submodule is configured to calculate a loss function of an initial super-resolution model using the input image set and a restored image set corresponding to the input image set, and update parameters of the initial super-resolution model using a gradient descent method; The second updating submodule is configured to calculate the distance between the features of the input image set and the images in the restored image set based on the identity identification number, including: respectively calculating the distance between the features of images with the same identity identification number and the distance between the features of images with different identity identification numbers; calculating the loss function of the initial recognition model based on the distance between the features of the input image set and the images in the restored image set, including: when two images have the same identity identification number, the loss function is the square of the distance between the features of the two images; and using the gradient descent method to update the parameters of the initial recognition model.
8. The device according to claim 7, wherein: The first updating submodule comprises: A downsampling unit, configured to downsample the images in the input image set to obtain a downsampled image set; A restoration unit, configured to restore images in the downsampled image set using the initial super-resolution model to obtain a restored image set; The first calculation unit is configured to calculate the reconstruction loss of the initial super-resolution model based on the input image set and the restored image set.
9. The device according to claim 8, wherein: The second updating submodule includes: a merging unit, configured to merge the input image set, the downsampled image set and the restored image set to obtain a target image set; an extraction unit, configured to extract features of images in the target image set; A second calculation unit is configured to calculate the distance between the features of the images in the target image set; The third calculation unit is configured to calculate the bigram loss function of the initial recognition model based on the distance.
10. The device according to claim 7, wherein: The combined module comprises: The connection submodule is configured to connect the output end of the part before the loss function in the trained super-resolution model to the input end of the recognition model.
11. An image recognition device, comprising: A second acquisition module is configured to acquire an image to be recognized; The output module is configured to input the image to be recognized into an image recognition model and output a recognition result corresponding to the image to be recognized, wherein the image recognition model is obtained by the method for constructing an image recognition model as described in any one of claims 1 to 5.
12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
13. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method of any one of claims 1 to 6.
Citation Information
Patent Citations
Construction method, recognition method and construction device of fine-grained face recognition model
CN111368790A
Face recognition method and device, electronic equipment and storage medium
CN112052789A
Model training method and device, image recognition method and device, equipment and storage medium
CN112149756A
Training method of SRFlow super-resolution model and face recognition method
CN112507617A
Translation model training method, translation method, electronic equipment and storage medium
CN112699690A
