Image registration method, device, and electronic device

Abstract

The present application discloses an image registration method, belonging to the field of computer technology, which helps to improve the efficiency of image registration. The image registration method disclosed in the embodiments of the present application includes: obtaining a preset number of images to be registered; obtaining the image category of each of the images to be registered; according to the image category of each of the images to be registered, sorting the preset number of images to be registered in the order of preset image registration categories from front to back to obtain a sequence of images to be registered composed of the preset number of images to be registered; and performing image registration based on the sequence of images to be registered. The image registration method disclosed in the embodiments of the present application can improve the efficiency of image registration by identifying the categories of images to be registered and adjusting the input order of the images.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technologies, and particularly to an image registration method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the increasingly wide application of Internet applications, the requirements for network security are also continuously increasing. Many network platforms require users to register and verify their identities before they can perform some operations on the network platform. For example, in a food delivery platform, users who need to register as food delivery riders are required to upload photos of themselves and their certificates in a certain order. After that, the registration application compares and identifies the photos uploaded by the users in sequence, and the registration can be successful only after the comparison and identification pass. In the prior art, the image comparison and identification module of the registration application has strict requirements on the types and order of the photos uploaded by users. It often happens that the registration fails due to the incorrect order of the photos uploaded by users, and users need to repeatedly perform the photo upload operation.

[0003] It can be seen that there is a problem of low registration efficiency in the identity registration system in the prior art. Summary of the Invention

[0004] Embodiments of the present application provide an image registration method, which helps to improve the registration efficiency of the identity registration system.

[0005] To solve the above problems, in a first aspect, embodiments of the present application provide an image registration method, including:

[0006] Obtaining a preset number of images to be registered;

[0007] Obtaining the image category of each of the images to be registered;

[0008] Sorting the preset number of images to be registered in ascending order according to the preset image registration category order based on the image category of each of the images to be registered, to obtain a sequence of images to be registered composed of the preset number of images to be registered;

[0009] Performing image registration based on the sequence of images to be registered.

[0010] In a second aspect, embodiments of the present application provide an image registration apparatus, including:

[0011] An image-to-be-registered acquisition module, configured to obtain a preset number of images to be registered;

[0012] An image category acquisition module, configured to obtain the image category of each of the images to be registered;

[0013] A sorting module, configured to sort the preset number of images to be registered in the order of the preset image registration categories from front to back according to the image categories of each of the images to be registered, so as to obtain a sequence of images to be registered composed of the preset number of images to be registered;

[0014] A registration module, configured to perform image registration based on the sequence of images to be registered.

[0015] In a third aspect, an embodiment of the present application further discloses an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the image registration method described in the embodiment of the present application is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the image registration method disclosed in the embodiment of the present application are implemented.

[0017] The image registration method disclosed in the embodiment of the present application can improve the efficiency of image registration by obtaining a preset number of images to be registered, obtaining the image category of each of the images to be registered, then sorting the preset number of images to be registered in the order of the preset image registration categories from front to back according to the image category of each of the images to be registered, so as to obtain a sequence of images to be registered composed of the preset number of images to be registered, and finally performing image registration based on the sequence of images to be registered.

[0018] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically describes the specific embodiments of the present application. Description of the Drawings

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0020] Figure 1 It is a flowchart of the image registration method according to Embodiment 1 of the present application;

[0021] Figure 2a It is one of the schematic diagrams of images to be registered of various categories in the image registration method according to Embodiment 1 of the present application;

[0022] Figure 2b It is the second schematic diagram of various types of images to be registered in the image registration method according to the first embodiment of the present application;

[0023] Figure 2c It is the third schematic diagram of various types of images to be registered in the image registration method according to the first embodiment of the present application;

[0024] Figure 3 It is the schematic diagram of model input in the image registration method according to the first embodiment of the present application;

[0025] Figure 4 It is the flowchart of the image registration method according to the second embodiment of the present application;

[0026] Figure 5 It is the first schematic diagram of the structure of the image registration device according to the third embodiment of the present application;

[0027] Figure 6 It is the second schematic diagram of the structure of the image registration device according to the third embodiment of the present application;

[0028] Figure 7 Schematically shows a block diagram of an electronic device for executing the method according to the present application; and

[0029] Figure 8 Schematically shows a storage unit for holding or carrying program code for implementing the method according to the present application. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0031] Embodiment 1

[0032] An image registration method disclosed in an embodiment of the present application, as Figure 1 shown, the method includes: Step 110 to Step 140.

[0033] Step 110, obtaining a preset number of images to be registered.

[0034] During the user identity registration process, in order to conduct identity verification, many application scenarios require users to provide face images, document images, and images including both face and document. For example, during the registration of logistics delivery personnel (such as food delivery riders, mail carriers, couriers, etc.), it is necessary to upload an image of the delivery personnel holding a document with their face facing forward (such as Figure 2a ), a front image of the document (such as Figure 2b)), a front face image (such as Figure 2c ); During the company registration application process, it is necessary to upload the legal person's front face image, the front image of the certificate, and the image of the front face holding the certificate.

[0035] In similar application scenarios above, some registration applications call the interface of the image acquisition device to guide the registered user to collect a preset number of images to be registered in real time (such as collecting three images of a food delivery rider, including: a front face image, a front image of the certificate, and an image of the front face holding the certificate), and some registration applications call the interface of the image management software (such as the system album) to guide the registered user to upload the preset number of images to be registered in real time.

[0036] The registration application can collect the above-mentioned preset number of images to be registered through the image acquisition device, or obtain the preset number of images to be registered uploaded by the user by calling the interface of the image management software. For example, the obtained images to be registered are represented as: pic1, pic2, and pic3.

[0037] In the embodiments of the present application, the specific implementation manner of obtaining the images to be registered is not limited.

[0038] Step 120, obtain the image category of each of the images to be registered.

[0039] In the prior art, the registration application will guide the registered user to collect or upload images of different image categories in sequence according to the interface of the preset image recognition engine. For example, during the registration process of a food delivery rider, the registration application will guide the user to collect in sequence: an image of the front face holding the certificate, a front image of the certificate, and a front face image. When the registration application collects the above images in sequence, it will label the collected images to be registered with their image categories in sequence and store them separately. After obtaining the preset number of images to be registered (such as the images to be registered pic1, pic2, and pic3), the image comparison and recognition module of the registration application will perform pairwise comparison on the images to be registered in the order of collection of the images to be registered, and perform user registration according to the comparison results. For example, by comparing the face H1 holding the certificate, the face H2 on the certificate, the front image of the certificate (such as Figure 2a )), the face H3 in the front image of the certificate (such as Figure 2b )), and the face H4 in the front face image (such as Figure 2c )). If the pairwise comparison results meet the preset conditions, it is determined that the image registration is passed.

[0040] In the prior art, the image comparison and recognition module sets different confidence threshold values for the comparison results of two images to be registered of different image categories. For example, the pairwise comparison categories and orders of images preset by the image comparison and recognition module are as follows: a front-face person holding a document image and a document front image, with a confidence threshold of T12; a front-face person holding a document image and a front-face person image, with a confidence threshold of T13; a document front image and a front-face person image, with a confidence threshold of T23, where the confidence threshold values T12, T23, and T13 correspond to different numerical values.

[0041] Since the registration application defaults that the user collects or uploads the images to be registered in the guided order, the image categories of the images to be registered collected or uploaded in sequence by the registration application are consistent with the image categories of the images to be registered input sequentially by the image comparison and recognition module, and the images to be registered collected or uploaded in sequence are input into the image comparison and recognition module for registration and recognition. However, if the user does not upload or collect the images to be registered of the corresponding categories in the guided order, the image comparison and recognition module will compare images with mismatched content and type. Since the content of the images to be registered of different image categories varies greatly, the comparison result will not meet the preset confidence threshold, thereby affecting the image registration result and causing the registration to fail. The user needs to repeat uploading or collecting the images to be registered, reducing the image registration efficiency.

[0042] In some embodiments of the present application, the image content is recognized through image recognition technology to determine the image category of each image to be registered. For example, several images of each image category can be collected in advance, and an image category recognition model for each image category is trained based on the several images of each image category. In a specific application, for each collected image to be registered (such as the images to be registered pic1, pic2, and pic3), it is input into the image category recognition model of each image category to determine the image category of each image to be registered.

[0043] In some preferred embodiments of the present application, the obtained preset number of images to be registered (such as the images to be registered pic1, pic2, and pic3) are used as an image sequence and input into a pre-trained image sequence category recognition model to determine the image category of each image to be registered in the image sequence. For example, obtaining the image category of each of the images to be registered includes: using the image sequence to be recognized composed of the preset number of images to be registered as the input of the pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model; determining the image category of each image in the image sequence to be recognized according to the image sequence category corresponding to the image sequence to be recognized. The following details the specific technical solution for combining and recognizing the preset number of images to be registered based on the image sequence to obtain the image category of each of the images to be registered.

[0044] In some embodiments of the present application, the preset number of images to be registered can be arranged in any order to form an image sequence to be recognized. For example, the obtained images to be registered pic1, pic2, and pic3 are arranged as the image sequence to be recognized: pic1pic2 pic3. Using the image sequence to be recognized composed of the preset number of images to be registered as the input of the pre-trained image sequence category recognition model includes: sequentially splicing the images to be registered in the image sequence to be recognized into one image (as shown in Figure 3 ), and inputting the spliced image into the pre-trained image sequence category recognition model; or separately extracting the preset image features of the images to be registered in the image sequence to be recognized, and sequentially splicing the preset image features of each image to be registered and then inputting them into the pre-trained image sequence category recognition model.

[0045] For the specific implementation manner of image splicing, refer to the prior art and will not be elaborated in the embodiments of the present application. The aforementioned preset image features can be texture features, color features, or other image features, and the present application does not make any limitations in this regard. For the specific implementation manner of image feature splicing, refer to the prior art and will not be elaborated in the embodiments of the present application.

[0046] Specifically, when implementing, the image sequence to be recognized composed of the preset number of images to be registered is input into the pre-trained image sequence category recognition model in the form of a spliced image or a spliced feature, which is determined according to the input requirements of the image sequence category recognition model. Taking the input data of the image sequence category recognition model as an image as an example, correspondingly, it is necessary to sequentially splice the images to be registered in the image sequence to be recognized into one image (as shown in Figure 3 ), and input the spliced image into the pre-trained image sequence category recognition model.

[0047] The pre-trained image sequence category recognition model extracts and maps features of the input image, and outputs probability values of the input image matching each preset image sequence category. Each of the probability values indicates whether the input image belongs to the image sequence category. Taking the preset image sequence categories including six first image sequence categories, respectively represented as C1, C2, C3, C4, C5, and C6 as an example, the image sequence category recognition model will output probability values corresponding to these six image sequence categories respectively. Then, the first image sequence category (such as C1) corresponding to the maximum probability value can be used as the image sequence category matched by the input image.

[0048] In some embodiments of the present application, according to the image sequence category corresponding to the image sequence to be recognized, determining the image category of each image in the image sequence to be recognized includes: determining the image category of each image in the image sequence to be recognized according to the preset corresponding relationship between the arrangement order of the image categories and the first image sequence categories. For example, in some embodiments of the present application, the arrangement order of the image categories corresponding to each of the first image sequence categories is pre-established. Taking the image categories including: front face holding a document image, document front image, and front face image as an example, assuming that the arrangement order of the graphic categories corresponding to the first image sequence category C1 is from front to back as: front face holding a document image, document front image, front face image. Further, it can be based on the arrangement order of the graphic categories corresponding to the first image sequence category C1 from front to back as: the first one, front face holding a document image; the second one, document front image; the third one, front face image. Thus, the image of each image in the image sequence to be recognized can be determined.

[0049] In some embodiments of the present application, before the step of using the image sequence to be recognized composed of the preset number of images to be registered as the input of the pre-trained image sequence category recognition model and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model, it further includes: determining the same number of first image sequence categories according to the arrangement number of the image categories, and determining the first image sequence categories corresponding to each arrangement order of the image categories one by one; obtaining several image sequences corresponding to each of the first image sequence categories according to the arrangement order of the image categories corresponding to each of the first image sequence categories, and constructing a training sample corresponding to the first image sequence category based on each image sequence; training the image sequence category recognition model based on the constructed training samples. That is, before using the image sequence category recognition model, it is first necessary to train the image sequence category recognition model.

[0050] For the convenience of readers to more clearly understand the beneficial effects brought by the technical solution of the present application, the technical means adopted in the technical solution will be described in detail below in combination with the training process of the image sequence category recognition model.

[0051] Taking the application scenario of food delivery riders for image registration as an example, the registration application usually requires food delivery riders to collect a front face holding a document image, a front document image, and a front face image in sequence. When determining the image category of the image to be registered, if each image to be registered is input into a pre-trained image category recognition model of a single image category for recognition respectively, and when determining the image category of each image to be registered respectively, due to the very similar image content of the front face holding a document image and the front face image, it is very easy to cause misrecognition. Therefore, in the preferred embodiment of the present application, the obtained preset number of images to be registered of human faces are combined in any order to form an image sequence to be recognized, and the preset number of images to be registered of human faces and are combined and recognized by the image sequence category recognition model, so as to reduce the misrecognition rate in a way that weakens the easily confused features. By recognizing the combination order of the image sequence to be recognized, the image category of each image to be registered in the image sequence to be recognized is further determined. Correspondingly, when training the image sequence category recognition model, a number of training samples corresponding to each image sequence category need to be constructed.

[0052] In some embodiments of the present application, different combination orders of all preset image categories of the image to be registered are respectively used as a first image sequence category. That is, the number of permutation types of all preset image categories is used as the number of first image sequence categories, and each first image sequence category corresponds to a permutation of all preset image categories. Still taking the images to be registered of a food delivery rider including three image categories: a front face holding a document image, a front document image, and a front face image as an example, hereinafter, for the convenience of description, the front face holding a document image is represented by the symbol "P1", the front document image is represented by the symbol "P2", and the front face image is represented by the symbol "P3". Then, according to the permutation formula A(3, 3), it can be obtained that the number of permutation types of these three image categories is six, for example, they are respectively represented as: P1 P2 P3, P1 P3 P2, P2 P1 P3, P2 P3 P1, P3 P1 P2, P3 P2 P1. Correspondingly, it can be determined that the first image sequence category includes six types, and each first image sequence category corresponds to a permutation type of the above three image categories. For example, if the six first image sequence categories are respectively represented by the symbols: C1, C2, C3, C4, C5, and C6, then the permutation order of the image categories corresponding to the first image sequence category C1 can be set as P1 (i.e., the front face holding a document image) P2 (i.e., the front document image) P3 (i.e., the front face image); the permutation order of the image categories corresponding to the first image sequence category C2 can be set as P1 P3 P2;...

[0053] In some embodiments of the present application, before training the image sequence category recognition model, it is also necessary to construct a number of training samples corresponding to each of the first image sequence categories. Among them, the sample label of each training sample is the category identifier of the first image sequence category; the sample data is an image sequence, and the arrangement order of the images in the image sequence matches the arrangement order of the image categories corresponding to the first image sequence category. For example, among the sample data of all training samples corresponding to the first image sequence category C1, the image category of the first image is P1 (i.e., a front-facing human face holding a document image), the image category of the second image is P2 (i.e., the front side of the document image), and the image category of the third image is P3 (i.e., a front-facing human face image).

[0054] In some embodiments of the present application, the training samples can be constructed by directly collecting the image sequences corresponding to various first image sequence categories. For example, the images of the above three image categories of each rider are collected in different orders to obtain a number of image sequences. However, collecting image sequences is time-consuming and a large number of training samples cannot be obtained.

[0055] In some embodiments of the present application, it is also possible to first obtain a number of images of each image category, and then construct the training samples by generating the image sequences corresponding to various first image sequence categories through image combination and sorting. Based on the requirement of combining and recognizing the preset number of to-be-registered face images in any order to form a to-be-recognized image sequence, when constructing the training samples of the model, in order to ensure the distribution balance of the training samples of each first image sequence category, that is, the training samples of the combination order of various image categories, and improve the model recognition accuracy, in some embodiments of the present application, the images of various image categories are regularly combined to generate new samples to increase the number of training samples and the sample distribution balance.

[0056] For example, in some embodiments of the present application, the step of obtaining a number of image sequences corresponding to each of the first image sequence categories according to the arrangement order of the image categories corresponding to each of the first image sequence categories, and constructing a training sample corresponding to the first image sequence category based on each image sequence includes: obtaining a number of images of each image category; enumerating the image combinations obtained by respectively selecting one image from the number of images of each image category; for each of the first image sequence categories, sorting the images in each image combination according to the arrangement order of the image categories corresponding to the first image sequence category to obtain the training sample corresponding to the first image sequence category.

[0057] Still taking the image categories including: frontal face holding a document image, front side of the document image, and frontal face image as an example, the frontal face holding a document image is represented by the symbol "P1", the front side of the document image is represented by the symbol "P2", and the frontal face image is represented by the symbol "P3". First, a number of frontal face holding a document images, a number of front side of the document images, and a number of frontal face images need to be obtained. In specific implementation, it is difficult to collect some categories of images. For example, for the frontal face image, due to the unbalanced samples, overfitting is likely to occur during the model training process. Therefore, in some embodiments of the present application, a sample combination method is adopted to regenerate "new samples".

[0058] Taking the symbol Pi_j to represent the image of the i-th image category of the j-th rider as an example, the technical solution for combining and generating training samples will be described in detail. Wherein, i and j are positive integers. Specifically in this embodiment, assuming that the above 3 types of images of 100 riders are used as the original samples, the value range of j is from 1 to 100, and i can take the values 1, 2, and 3. P1_1 represents the first type of image of rider 1 (i.e., the frontal face holding a document image), P2_1 represents the second type of image of rider 1 (i.e., the front side of the document image), and P3_1 represents the third type of image of rider 1 (i.e., the frontal face image). A number of images of each image category of each rider can be collected. For example, 10 images of the first category (i.e., the frontal face holding a document image) of rider 1, 10 images of the second category (i.e., the front side of the document image) of rider 1, and 10 images of the third category (i.e., the frontal face image) of rider 1 are collected. According to the same method, a number of images of each image category of other riders are collected. Finally, a number of images of each image category can be obtained.

[0059] To increase the sample quantity and further improve the recognition accuracy of the trained model, next, a number of images of each image category are combined to generate a number of image sequences. For example, select an image P1_1 from the first image category, select an image P2_1 from the second image category, and form an image sequence with each image in the third image category respectively. In this way, if there are N images in the third image category, N image sequences corresponding to the first image sequence category C1 (i.e., the arrangement order of the corresponding image categories is P1 P2 P3) will be obtained. Select an image P1_1 from the first image category, select an image P2_2 from the second image category, and form an image sequence with each image in the third image category respectively, and N more image sequences corresponding to the first image sequence category C1 (i.e., the arrangement order of the corresponding image categories is P1 P2 P3) will be obtained. According to this method, a number of image sequences corresponding to the first image sequence category will be obtained. In the same way, a number of image sequences corresponding to each first image sequence category can be obtained.

[0060] The above method covers all permutations after selecting one image from each of the three image categories to form an image combination.

[0061] For the image sequence corresponding to each first image sequence category, each image sequence is used as the sample data of a training sample, and the category identifier of the corresponding first image sequence category of the image sequence is used as the sample label of the training sample, so as to construct a piece of training data, and several training samples corresponding to the first image sequence category can be obtained.

[0062] After determining several training samples corresponding to each first image sequence category, start training the image sequence category recognition model based on the constructed training samples. In some embodiments of the present application, the image sequence category recognition model is implemented using the ResNet50 network structure. In other embodiments of the present application, the image sequence category recognition model can also adopt other multi-classification network structures. The present application does not limit the specific network structure adopted by the image sequence category recognition model, as long as it can output multi-classification results for the input data.

[0063] For the specific implementation manner of training the image sequence category recognition model based on the constructed training samples, refer to the prior art, and it will not be elaborated in the embodiments of the present application.

[0064] Step 130, according to the image category of each of the to-be-registered images, sort the preset number of to-be-registered images in the order of the preset image registration category from front to back, so as to obtain a to-be-registered image sequence composed of the preset number of to-be-registered images.

[0065] After determining the image category of each to-be-registered image, for example, the image categories of the to-be-registered images pic1, pic2, and pic3 are in turn: front face holding a document image P1, front face image P3, and front side of the document image P2, and the preset image registration category order (that is, the default image registration category order of the image comparison and recognition module in the registration application) is: front face holding a document image P1, front side of the document image P2, and front face image P3. Then, rearrange the to-be-registered images pic1, pic2, and pic3 to obtain the to-be-registered image sequence pic1, pic3, pic2, that is, adjust the arrangement order of the obtained to-be-registered images to be consistent with the input requirements of the image comparison and recognition module.

[0066] Step 140, perform image registration based on the to-be-registered image sequence.

[0067] Finally, input the to-be-registered image sequence obtained after adjusting the order into the image comparison and recognition module, and perform comparison and recognition of the to-be-registered images through the image comparison and recognition module, so as to complete image registration.

[0068] The image registration method disclosed in the embodiments of the present application obtains a preset number of images to be registered, and obtains the image category of each of the images to be registered. Then, according to the image category of each of the images to be registered, the preset number of images to be registered are sorted in ascending order according to the preset image registration category order, obtaining a sequence of images to be registered composed of the preset number of images to be registered. Finally, image registration is performed based on the sequence of images to be registered, which can improve the efficiency of image registration.

[0069] Specifically, the image registration method disclosed in the embodiments of the present application adds an operation for identifying the category and correcting the order of the images to be registered in the image registration link of the existing identity registration system, so as to automatically correct errors in the case of abnormal types of uploaded images, thereby improving the automatic passing rate. There is no strict requirement for the order in which users collect or upload images to be registered, which improves the usability of the registration system. A large amount of experimental data shows that when using the image registration method disclosed in the embodiments of the present application for rider registration, the automatic registration success rate is increased by 5 percentage points compared with the prior art. On the other hand, in the training process, sample combinations are used as training sample units to make up for the shortage of some data types in the initial stage of business development, enrich the samples, and further improve the accuracy of image category recognition.

[0070] Embodiment 2

[0071] In the training process of the image sequence category recognition model of the image registration method disclosed in the embodiments of the present application, the image sequence category further includes: a second image sequence category. After the step of determining the same number of first image sequence categories according to the number of arrangements of the image categories and determining the first image sequence categories corresponding to each arrangement order of the image categories, the method further includes: determining a second image sequence category different from each of the first image sequence categories, and determining that the arrangement order of the image categories corresponding to the second image sequence category is an arrangement order including at least repeated image categories. Wherein, the second image sequence category corresponds to an abnormal image sequence. For example, in the image sequence, at least two images have the same category. Referring to the description in Embodiment 1, the second image sequence category can be represented by the symbol C7. Specifically, for the image categories described in Embodiment 1, an image sequence composed of a front face with a document in hand image P1, a front face image P3, and a front face with a document in hand image P1 belongs to the second image sequence category; another example is an image sequence composed of a front face with a document in hand image P1 and two front face images P3 belongs to the second image sequence category.

[0072] When constructing the training samples, according to at least two images arbitrarily selected from several images of any one of the said image categories and the images selected from several images of the remaining said image categories, construct the training samples corresponding to the second image sequence category. For example, arbitrarily select two images from several images of the frontal face holding a document image P1 category and arbitrarily select one image from several images of the frontal face image P3 category, and arrange them in any order to generate multiple training samples. The sample labels of these multiple training samples can be set as C7 for example.

[0073] Correspondingly, training the image sequence category recognition model based on the constructed training samples includes: training the image sequence category recognition model based on the training samples corresponding to each of the first image sequence categories and the training samples corresponding to the second image sequence category. Finally, based on the training samples corresponding to the first image sequence category (i.e., the normal image sequence category) and the training samples corresponding to the second image sequence category (i.e., the abnormal image sequence category), train the image sequence category recognition model.

[0074] Correspondingly, after the step of using the image sequence to be recognized composed of the preset number of images to be registered as the input of the pre-trained image sequence category recognition model and identifying the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model, it further includes: determining that the image category of the images in the image sequence to be recognized is abnormal according to the image sequence category corresponding to the image sequence to be recognized. For example, if the image sequence category matches the first image sequence category, determine the image category of each image in the image sequence to be recognized according to the image sequence category corresponding to the image sequence to be recognized; if the image sequence category matches the second image sequence category, determine that the image category of the input images to be registered is abnormal. As mentioned above, during the model training process, if the image sequence categories corresponding to the training samples include seven, the sample labels of the training samples correspond to seven image sequence categories (for example, six first image sequence categories represented by C1 to C6 and one second image sequence category represented by C7), the output of the image sequence category recognition model will correspond to the classification probability values of seven image sequence categories.

[0075] Further, according to the classification probability values corresponding to the seven image sequence categories output by the image sequence category recognition model, it is possible to determine which image sequence category the input image sequence to be recognized corresponds to. For example, when the classification probability value corresponding to the first image sequence category C1 output by the image sequence category recognition model satisfies the preset probability value condition (such as the probability value is the largest), it is determined that the input image sequence to be recognized corresponds to the first image sequence category C1. Then, further, according to the arrangement order of the image categories corresponding to the first image sequence category, the image categories of the to-be-registered images arranged in sequence in the image sequence to be recognized can be determined. For another example, when the classification probability value corresponding to the second image sequence category C7 output by the image sequence category recognition model satisfies the preset probability value condition (such as the probability value is the largest), it is determined that the input image sequence to be recognized corresponds to the second image sequence category C7, and it can be determined that the to-be-registered images in the image sequence to be recognized include multiple images of the same category or the images do not meet the requirements.

[0076] Correspondingly, as Figure 4 shown, after the step of obtaining the image category of each to-be-registered image, the following steps 150 and 160 are further included.

[0077] Step 150. Determine whether the image categories of the images in the image sequence to be recognized are abnormal. If so, execute step 160; otherwise, execute step 130.

[0078] If the image sequence to be recognized belongs to the second image sequence category, it means that the to-be-registered images in the image sequence to be recognized include at least two images of the same image category, which do not meet the requirements of image registration. Re-sorting is no longer performed, and the user needs to re-collect or upload the to-be-registered images.

[0079] Step 160. If it is determined that the image categories of the images in the image sequence to be recognized are abnormal, output a prompt message indicating that the to-be-registered image category is incorrect, and return to the step of obtaining a preset number of to-be-registered images.

[0080] When it is determined that the image categories of the images in the image sequence to be recognized are abnormal, the registration application will output a prompt message indicating that the to-be-registered image category is abnormal, so that the registered user can re-collect or upload the to-be-registered images.

[0081] The image registration method disclosed in the embodiments of the present application can improve the recognition accuracy of the trained model by balancing the sample distribution by constructing training samples of abnormal image category sequences and training an image sequence category recognition model based on the training samples of normal image sequence categories and abnormal image sequence categories. Further, in the prior art, when the image comparison and recognition module fails to pass the comparison of the image to be registered, it only prompts the user to re-collect or upload the image, and cannot give an accurate prompt. However, the image registration method disclosed in the embodiments of the present application can identify the image to be registered in the abnormal image category and give an accurate prompt to the registered user, which helps the registered user to improve the registration efficiency and can further improve the user experience.

[0082] Embodiment III

[0083] An image registration device disclosed in the embodiments of the present application, as Figure 5 shown, the device includes:

[0084] A to-be-registered image acquisition module 510, configured to acquire a preset number of images to be registered;

[0085] An image category acquisition module 520, configured to acquire the image category of each of the images to be registered;

[0086] A sorting module 530, configured to sort the preset number of images to be registered in the order of the preset image registration category from front to back according to the image category of each of the images to be registered, so as to obtain a to-be-registered image sequence composed of the preset number of images to be registered;

[0087] A registration module 540, configured to perform image registration based on the to-be-registered image sequence.

[0088] In some embodiments of the present application, the image category acquisition module 520 is further configured to:

[0089] Take the to-be-identified image sequence composed of the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and identify the image sequence category corresponding to the to-be-identified image sequence through the image sequence category recognition model;

[0090] Determine the image category of each image in the to-be-identified image sequence according to the image sequence category corresponding to the to-be-identified image sequence.

[0091] In some embodiments of the present application, the image category acquisition module 520 is further configured to:

[0092] Determine that the image category of the images in the to-be-identified image sequence is abnormal according to the image sequence category corresponding to the to-be-identified image sequence;

[0093] AsFigure 6 As shown, the device further includes:

[0094] An exception handling module 550, configured to output a prompt message indicating an error in the image category to be registered and return to the image acquisition module for the image to be registered if it is determined that the image category of the images in the image sequence to be recognized is abnormal.

[0095] In some embodiments of the present application, as Figure 6 shown, before the step of using the image sequence to be recognized composed of the preset number of images to be registered as the input of the pre-trained image sequence category recognition model and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model, it includes: a training sample construction module 560 and a model training module 570,

[0096] wherein, the training sample construction module 560 is configured to determine the same number of first image sequence categories according to the arrangement number of the image categories, and determine the first image sequence categories corresponding to each arrangement order of the image categories; and,

[0097] According to the arrangement order of the image categories corresponding to each of the first image sequence categories, obtain a plurality of image sequences corresponding to each of the first image sequence categories, and construct a training sample corresponding to each first image sequence category based on each image sequence;

[0098] The model training module 570 is configured to train the image sequence category recognition model based on the constructed training samples.

[0099] In some embodiments of the present application, the step of obtaining a plurality of image sequences corresponding to each of the first image sequence categories according to the arrangement order of the image categories corresponding to each of the first image sequence categories and constructing a training sample corresponding to each first image sequence category based on each image sequence includes:

[0100] Obtain a plurality of images of each of the image categories;

[0101] Enumerate the image combinations obtained by respectively selecting one image from the plurality of images of each of the image categories;

[0102] For each of the first image sequence categories, sort the images in each of the image combinations according to the arrangement order of the image categories corresponding to the first image sequence category to obtain the training sample corresponding to the first image sequence category.

[0103] In some embodiments of the present application, the training sample construction module 560 is further configured to, after determining the same number of first image sequence categories according to the arrangement quantity of the image categories, and determining the first image sequence categories corresponding to each arrangement order of the image categories, determine a second image sequence category different from each of the first image sequence categories, and determine that the arrangement order of the image categories corresponding to the second image sequence category is an arrangement order including at least repeated image categories; and,

[0104] Construct a training sample corresponding to the second image sequence category according to at least two images arbitrarily selected from several images of any one of the image categories and images selected from several images of the remaining image categories;

[0105] Correspondingly, the model training module 570 is further configured to train the image sequence category recognition model based on the training samples corresponding to each of the first image sequence categories and the training sample corresponding to the second image sequence category.

[0106] The image registration device disclosed in the embodiments of the present application is used to implement the image registration method described in Embodiment 1 or Embodiment 2 of the present application. The specific implementation manners of the modules of the device will not be elaborated here, and reference may be made to the specific implementation manners of the corresponding steps in the method embodiments.

[0107] The image registration device disclosed in the embodiments of the present application can improve the efficiency of image registration by obtaining a preset number of images to be registered, obtaining the image category of each image to be registered, then sorting the preset number of images to be registered in the order of the preset image registration category from front to back according to the image category of each image to be registered, obtaining an image sequence to be registered composed of the preset number of images to be registered, and finally performing image registration based on the image sequence to be registered.

[0108] Specifically, the image registration device disclosed in the embodiments of the present application adds an operation of category recognition and order correction of images to be registered in the image registration link of the existing identity registration system, so as to automatically correct the situation of abnormal types of uploaded images, thereby improving the automatic passing rate. There is no strict requirement for the order in which users collect or upload images to be registered, which improves the usability of the registration system. A large number of experimental data show that when using the image registration method disclosed in the embodiments of the present application for rider registration, the automatic registration success rate is increased by 5 percentage points compared with the prior art. On the other hand, during the training process, sample combinations are used as training sample units to make up for the shortage of some data types in the initial stage of business development, enrich the samples, and further improve the accuracy of image category recognition.

[0109] Furthermore, in the prior art, when the image comparison and recognition module fails to pass the comparison of the image to be registered, it only prompts the user to re-collect or upload the image intelligently, and cannot give an accurate prompt. However, the image registration method disclosed in the embodiments of the present application can identify the image to be registered with an abnormal image category and give an accurate prompt to the registered user, which helps the registered user improve the registration efficiency and can further improve the user experience.

[0110] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is the difference from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the related parts, refer to the partial description of the method embodiments.

[0111] The above provides a detailed introduction to an image registration method and device provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0113] Each component embodiment of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of the present application. The present application can also be implemented as a device or device program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0114] For example, Figure 7An electronic device capable of implementing the method according to the present application is shown. The electronic device may be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. Traditionally, the electronic device includes a processor 710, a memory 720, and program code 730 stored on the memory 720 and executable on the processor 710. When the processor 710 executes the program code 730, the method described in the above embodiments is implemented. The memory 720 may be a computer program product or a computer-readable medium. The memory 720 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. The memory 720 has a storage space 7201 for the program code 730 of a computer program for executing any method step in the above methods. For example, the storage space 7201 for the program code 730 may include respective computer programs for implementing various steps in the above methods. The program code 730 is computer-readable code. These computer programs may be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, compact discs (CDs), memory cards, or floppy disks. The computer program includes computer-readable code, which when running on the electronic device, causes the electronic device to execute the method according to the above embodiments.

[0115] An embodiment of the present application also discloses a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the image registration method described in Embodiment 1 or Embodiment 2 of the present application are implemented.

[0116] Such a computer program product may be a computer-readable storage medium, which may have storage segments, storage spaces, etc. arranged similarly to the memory 720 in the Figure 7 shown electronic device. The program code may be stored in the computer-readable storage medium in a suitably compressed form, for example. The computer-readable storage medium is generally a portable or fixed storage unit as referred to in Figure 8 the reference. Generally, the storage unit includes computer-readable code 730', which is code read by the processor. When these codes are executed by the processor, the respective steps in the method described above are implemented.

[0117] As used herein, the terms "one embodiment", "an embodiment", or "one or more embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. In addition, note that the examples of the phrase "in one embodiment" herein do not necessarily all refer to the same embodiment.

[0118] In the description provided herein, numerous specific details are set forth. However, it will be understood that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0119] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image registration method, characterized in that, it includes: Obtain a preset number of images to be registered; Obtain the image category of each of the images to be registered; According to the image category of each of the images to be registered, sort the preset number of images to be registered in the order of the preset image registration category from front to back, and obtain a sequence of images to be registered composed of the preset number of images to be registered; Perform image registration based on the sequence of images to be registered; Among them, the step of obtaining the image category of each of the images to be registered includes: Take the sequence of images to be recognized composed of the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and identify the image sequence category corresponding to the sequence of images to be recognized through the image sequence category recognition model; Determine the image category of each image in the sequence of images to be recognized according to the image sequence category corresponding to the sequence of images to be recognized; Among them, after the step of taking the sequence of images to be recognized composed of the preset number of images to be registered as the input of a pre-trained image sequence category recognition model and identifying the image sequence category corresponding to the sequence of images to be recognized through the image sequence category recognition model, it further includes: Determine that the image category of the images in the sequence of images to be recognized is abnormal according to the image sequence category corresponding to the sequence of images to be recognized; After the step of obtaining the image category of each of the images to be registered, it further includes: If it is determined that the image category of the images in the sequence of images to be recognized is abnormal, output a prompt message indicating that the image category to be registered is incorrect, and return to the step of obtaining a preset number of images to be registered; Among them, before the step of taking the sequence of images to be recognized composed of the preset number of images to be registered as the input of a pre-trained image sequence category recognition model and identifying the image sequence category corresponding to the sequence of images to be recognized through the image sequence category recognition model, it includes: According to the arrangement quantity of the image categories, determine the same number of first image sequence categories, and determine the first image sequence categories corresponding to each arrangement order of the image categories one by one; According to the arrangement order of the image categories corresponding to each of the first image sequence categories, obtain several image sequences corresponding to each of the first image sequence categories, and construct a training sample corresponding to the first image sequence category based on each image sequence; Train the image sequence category recognition model based on the constructed training samples.

2. The method according to claim 1, characterized in that, The step of obtaining several image sequences corresponding to each of the first image sequence categories according to the arrangement order of the image categories corresponding to each of the first image sequence categories and constructing a training sample corresponding to the first image sequence category based on each image sequence includes: Obtain several images of each of the image categories; Enumerate the image combinations obtained by respectively selecting one image from several images of each of the image categories; For each of the first image sequence categories, sort the images in each of the image combinations in the arrangement order of the image categories corresponding to the first image sequence category to obtain the training samples corresponding to the first image sequence category.

3. The method according to claim 1, wherein, after the step of determining the same number of first image sequence categories according to the number of arrangements of the image categories and determining the first image sequence categories corresponding to each arrangement order of the image categories, further comprising: determining a second image sequence category different from each of the first image sequence categories, and determining that the arrangement order of the image categories corresponding to the second image sequence category is an arrangement order including at least repeated image categories; constructing the training samples corresponding to the second image sequence category according to at least two images arbitrarily selected from several images of any one of the image categories and the images selected from several images of the remaining image categories; the step of training the image sequence category recognition model based on the constructed training samples includes: training the image sequence category recognition model based on the training samples corresponding to each of the first image sequence categories and the training samples corresponding to the second image sequence category.

4. An image registration device, wherein, comprising: a to-be-registered image acquisition module for acquiring a preset number of to-be-registered images; an image category acquisition module for acquiring the image category of each of the to-be-registered images; a sorting module for sorting the preset number of to-be-registered images in the order of a preset image registration category from front to back according to the image category of each of the to-be-registered images to obtain a to-be-registered image sequence composed of the preset number of to-be-registered images; a registration module for performing image registration based on the to-be-registered image sequence; wherein, the image category acquisition module is further configured to: take the to-be-identified image sequence composed of the preset number of to-be-registered images as the input of a pre-trained image sequence category recognition model, and identify the image sequence category corresponding to the to-be-identified image sequence through the image sequence category recognition model; determine the image category of each image in the to-be-identified image sequence according to the image sequence category corresponding to the to-be-identified image sequence; wherein, the image category acquisition module is further configured to: determine that the image categories of the images in the to-be-identified image sequence are abnormal according to the image sequence category corresponding to the to-be-identified image sequence; after acquiring the image category of each of the to-be-registered images, the image category acquisition module is further configured to: if it is determined that the image categories of the images in the to-be-identified image sequence are abnormal, output a prompt message indicating an error in the to-be-registered image category, and return to the step of acquiring the preset number of to-be-registered images; wherein, before taking the to-be-identified image sequence composed of the preset number of to-be-registered images as the input of a pre-trained image sequence category recognition model and identifying the image sequence category corresponding to the to-be-identified image sequence through the image sequence category recognition model, the image category acquisition module is further configured to: Determine the same number of first image sequence categories according to the number of arrangements of the image categories, and determine the first image sequence categories corresponding one by one to each arrangement order of the image categories; According to the arrangement order of the image categories corresponding to each of the first image sequence categories, obtain a number of image sequences corresponding to each of the first image sequence categories, and construct a training sample corresponding to the first image sequence category based on each image sequence; Train the image sequence category recognition model based on the constructed training samples.

5. The apparatus according to claim 4, wherein, the apparatus further comprises: An exception handling module, configured to output a prompt message indicating an error in the image category to be registered and return to the image acquisition module for the image to be registered if it is determined that an abnormality occurs in the image category of the images in the image sequence to be recognized.

6. An electronic device, comprising a memory, a processor, and program code stored on the memory and executable on the processor, wherein, when the processor executes the program code, the image registration method according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium, on which program code is stored, wherein, when the program code is executed by a processor, the steps of the image registration method according to any one of claims 1 to 3 are implemented.

Images