Image recognition and model processing method, device, equipment, medium and program product

Abstract

The invention relates to an image recognition and model processing method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a plurality of sample images with different source parties; the plurality of sample images comprise images obtained by shooting a target part of the sample object by different image acquisition devices; extracting features of the plurality of sample images through a multi-stream feature extractor of a to-be-trained model, and obtaining sample fusion features according to the extracted features; different feature extractors are used for processing sample images of different source parties; performing multi-scale feature extraction on the sample fusion features through a multi-scale network of the to-be-trained model to obtain sample target features of the target part; and training the to-be-trained model according to the sample target features of the target part and the training labels corresponding to the plurality of sample images, and obtaining a feature extraction model after the training is completed. The method can improve the accuracy of image feature extraction by the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image recognition and model processing method, apparatus, device, medium, and program product. Background Technology

[0002] With the development of artificial intelligence technology, it is being applied more and more widely in various object identification scenarios. For example, a trained model is used to extract features from an image to be identified, and these features are compared with those pre-stored in a database to determine the object's identity.

[0003] However, in cross-device recognition scenarios, where the devices that capture images pre-stored in the database and the devices that capture the image to be recognized are different, the resolution, color, and brightness of the two images differ due to the influence of the capturing devices. When traditionally trained models extract features from these images, the interference from the capturing devices results in poor accuracy, leading to a lower probability of successful feature matching. Therefore, improving the accuracy of image feature extraction in cross-device recognition scenarios has become a pressing issue. Summary of the Invention

[0004] Therefore, it is necessary to provide an image recognition and model processing method, apparatus, device, medium, and program product that can improve the accuracy of the model in extracting image features, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a model processing method. The method includes:

[0006] Acquire multiple sample images from different sources; these multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object;

[0007] Features from multiple sample images are extracted using a multi-stream feature extractor of the model to be trained, and sample fusion features are obtained based on the extracted features; different feature extractors are used to process sample images from different sources;

[0008] Multi-scale feature extraction of sample fusion features is performed using a multi-scale network of the model to be trained, so as to obtain the sample target features of the target part.

[0009] Based on the target features of the target area and the training labels corresponding to multiple sample images, the model to be trained is trained, and the feature extraction model is obtained after training.

[0010] In one embodiment, the step of performing cross-device style conversion on multiple first sample images to obtain multiple second sample images includes: for each first sample image, inputting the first sample image into a preset network to obtain a second sample image output by the preset network; the second sample image has a different device style from the corresponding first sample image, and the preset network includes at least one of a generation network or a diffusion network.

[0011] In one embodiment, obtaining the sample target features of the target region based on the attention processing result includes: extracting features from the attention processing result through a feature extraction layer of at least one scale to obtain the sample target features of the target region.

[0012] Secondly, this application provides an image recognition method. The method includes:

[0013] Acquire the image to be processed; the image to be processed includes images obtained by capturing the target part of the target object;

[0014] The feature extraction model is invoked to process the image to be processed and obtain the target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Different feature extractors are used to process sample images from different sources, and the multi-scale network is used to perform multi-scale feature extraction on the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor.

[0015] Identity recognition is performed based on target features to obtain the identity recognition result of the target object.

[0016] In one embodiment, the step of performing identity recognition based on the target feature to obtain the identity recognition result of the target object includes: comparing the target feature with each stored feature in the database; if a stored feature matching the target feature is obtained based on the comparison result, then the identity identifier corresponding to the stored feature is used as the identity identifier of the target object.

[0017] In one embodiment, the method further includes: if the identity recognition result indicates that the identity of the target object has been verified, then controlling the IoT device to perform a preset operation; or, if the identity recognition result includes an identity identifier, then performing a resource transfer operation based on the identity identifier.

[0018] Thirdly, this application also provides a model processing apparatus. The apparatus includes:

[0019] The acquisition module is used to acquire multiple sample images from different sources; these multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object;

[0020] The first feature extraction module is used to extract features from multiple sample images through the multi-stream feature extractor of the model to be trained, and to obtain sample fusion features based on the extracted features; different feature extractors are used to process sample images from different sources;

[0021] The second feature extraction module is used to extract multi-scale features from the sample fusion features through the multi-scale network of the model to be trained, so as to obtain the sample target features of the target part.

[0022] The training module is used to obtain a feature extraction model after training, based on the target features of the target area and the training labels corresponding to multiple sample images.

[0023] Fourthly, this application also provides an image recognition device. The device includes:

[0024] The acquisition module is used to acquire images to be processed; the images to be processed include images obtained by capturing images of the target parts of the target object;

[0025] The processing module is used to call the feature extraction model to process the image to be processed and obtain the target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Different feature extractors are used to process sample images from different sources, and the multi-scale network is used to perform multi-scale feature extraction on the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor.

[0026] The identification module is used to identify the target based on its characteristics and obtain the identification result of the target object.

[0027] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first or second aspects above.

[0028] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of the first or second aspects above.

[0029] In a seventh aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first or second aspects above.

[0030] The aforementioned image recognition and model processing methods, apparatuses, devices, media, and program products acquire multiple sample images from different sources, including images captured by different image acquisition devices of the target area of ​​the sample object. Features from these multiple sample images are then extracted using a multi-stream feature extractor of the model to be trained. Different feature extractors are used to process sample images from different sources, allowing each feature extractor to focus on extracting high-quality features from its corresponding source image. Sample fusion features are then obtained based on the extracted features. Multi-scale feature extraction is performed on these sample fusion features using a multi-scale network of the model to be trained to obtain the target features of the target area. This allows for the extraction of multi-scale features from sample images obtained from different sources (different devices), ensuring that these features match across different sources. This helps to achieve consistency of target area features under the specific influences (such as resolution) brought about by different device sources. Training the model to be trained using the target features extracted in the above manner and the training labels corresponding to the multiple sample images enables the model to learn key features unaffected by specific image acquisition devices, thereby improving the accuracy of image feature extraction and ultimately enhancing the model's generalization ability. When using the feature extraction model obtained after training for identity recognition, the feature extraction model can extract key features from the image to be processed that are not affected by specific image acquisition devices, thereby improving the accuracy of identity recognition. Attached Figure Description

[0031] Figure 1 This is a diagram illustrating the application environment of the model processing method in one embodiment;

[0032] Figure 2 This is a flowchart illustrating a model processing method in one embodiment;

[0033] Figure 3 Here is a diagram of the ResNet50 architecture in one embodiment;

[0034] Figure 4 Here is a diagram of the CycleGAN architecture in one embodiment;

[0035] Figure 5 This is an architecture diagram of a multi-stream feature extractor in one embodiment;

[0036] Figure 6 Here is an architecture diagram of a multi-stream feature extractor in another embodiment;

[0037] Figure 7 This is the architecture of the attention processing layer in one embodiment;

[0038] Figure 8 This is the architecture of the model to be trained in one embodiment;

[0039] Figure 9 This is a flowchart illustrating an image recognition method in one embodiment;

[0040] Figure 10 This is a structural block diagram of the model processing device in one embodiment;

[0041] Figure 11 This is a structural block diagram of an image recognition device in one embodiment;

[0042] Figure 12 This is an internal structural diagram of a computer device in one embodiment;

[0043] Figure 13 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] With the development of artificial intelligence technology, it is being used more and more widely in various object identification scenarios. For example, a trained deep learning network extracts features from an image to be identified and compares these features with those of images pre-stored in a database to determine the object's identity. The image to be identified may include images of faces, palm prints, or fingerprints.

[0046] However, in cross-device recognition scenarios, where the device that captures the images pre-stored in the database and the device that captures the image to be recognized are two different devices, the resolution, color, and brightness of the two images are different due to the influence of the capturing device. When the model trained in traditional technology extracts features from the image, the extracted features contain interference information due to the influence of the capturing device, resulting in poor accuracy and a reduced success rate in feature comparison.

[0047] Therefore, improving the accuracy of image feature extraction by the model in cross-device recognition scenarios has become an urgent problem to be solved. The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments.

[0048] The model processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be placed in the cloud or on another server. This model processing method can be executed on terminal 102 or server 104, or it can be implemented through the interaction between terminal 102 and server 104. Taking execution on server 104 as an example, server 104 acquires multiple sample images from different sources; these multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object; features of the multiple sample images are extracted using a multi-stream feature extractor of the model to be trained, and sample fusion features are obtained based on the extracted features; different feature extractors are used to process sample images from different sources; multi-scale feature extraction is performed on the sample fusion features using a multi-scale network of the model to be trained to obtain the sample target features of the target part; the model to be trained is trained based on the difference between the sample target features of the target part and the standard features, and a feature extraction model is obtained after training.

[0049] Terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, portable wearable devices, and aircraft. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc.

[0050] Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. Terminal 102 and server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0051] In one embodiment, such as Figure 2 As shown, a model processing method is provided, which is applied to a computer device (the computer device may specifically be...) Figure 1 Taking terminal 102 or server 104 as an example, the following steps are included:

[0052] Step 202: Obtain multiple sample images from different sources; the multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object.

[0053] The target body part of the sample object refers to the part that can be used to uniquely identify the sample object, such as the palm, face, eyes, or fingers. The image obtained by taking a picture of the target body part of the sample object can be a palm print image, a facial image, an iris image, or a fingerprint image, etc.

[0054] Image acquisition devices include, but are not limited to, mobile terminals used for identity registration using target body parts, payment devices that scan target body parts in payment scenarios (such as facial recognition payment devices, hand recognition payment devices, fingerprint payment devices, and iris recognition devices), boarding devices that scan target body parts in ride-hailing scenarios (such as facial recognition boarding devices, hand recognition boarding devices, fingerprint boarding devices, and iris recognition devices), and access control devices that scan target body parts in door opening and closing scenarios (facial recognition access control devices, hand recognition access control devices, fingerprint access control devices, and iris recognition devices).

[0055] Optionally, multiple sample images may consist only of images captured by different image acquisition devices of the target area of ​​the sample object. For example, multiple sample images may include an image of the palm print of the sample object captured by image acquisition device X (mobile phone) and an image of the palm print of the sample object captured by image acquisition device Y (palm payment device). In this case, the sources of the different sample images are distinguished by the type of image acquisition device.

[0056] In other embodiments, the multiple sample images include not only images captured by different image acquisition devices of the target area of ​​the sample object, but also images obtained by performing cross-device style conversion on images of the target area of ​​the sample object captured by different image acquisition devices. Cross-device style conversion refers to changing an image from one device style to another. In this case, the sources of different sample images are distinguished not only by the type of image acquisition device, but also by the device style simulated during cross-device style conversion.

[0057] For example, multiple sample images include: image 1 taken by image acquisition device X, image 2 taken by image acquisition device Y, image 3 obtained by cross-device style conversion of image 1 to have the same style as image acquisition device Y, and image 3 obtained by cross-device style conversion of image 2 to have the same style as image acquisition device X.

[0058] Optionally, the computer device can acquire multiple sample images from different sources in different public datasets. Images in different public datasets are taken by different image acquisition devices, while images belonging to the same public dataset are taken by the same image acquisition device. One type of image acquisition device corresponds to one source.

[0059] In other embodiments, the computer device acquires multiple first sample images from different public datasets, i.e., images captured by different types of image acquisition devices. The computer device then performs cross-device style conversion on each of the multiple first sample images to obtain multiple second sample images, i.e., images with different device-like styles. These multiple first sample images and multiple second sample images are then used as multiple first sample images to obtain multiple sample images from different sources.

[0060] Step 204: Extract features from multiple sample images using the multi-stream feature extractor of the model to be trained, and obtain sample fusion features based on the extracted features; different feature extractors are used to process sample images from different sources.

[0061] Here, the model to be trained refers to the feature extraction model to be trained. The structure of the model to be trained can be a combination of multiple deep learning networks or a variation of a certain deep learning network. For example, the model to be trained can be a model with the AGW model as the baseline network. The AGW model is a model with a Residual Network-50 (ResNet50) as the backbone network, which adds three blocks: A (Non-local Attention Block), G (Generalized-mean Pooling), and W (Weighted Regularization Triplet loss) to the ResNet50.

[0062] Reference Figure 3 The ResNet50 architecture diagram shown is illustrated. ResNet50 consists of five stages: stage 0, stage 1, stage 2, stage 3, and stage 4. Stage 0 can be viewed as preprocessing the input. Stages 1, 2, 3, and 4 are all composed of bottleneck layers. Stage 1 has 3 bottleneck layers, while stages 2, 3, and 4 have 4, 6, and 3 bottleneck layers, respectively. Figure 3 In this example, (3, 244, 244) refers to the input having 3 channels, a height of 244, and a width of 244, respectively, i.e., (C, H, W). When the height and width of the input are equal, it is represented by (C, W, W). In other words, C refers to the number of input channels, H refers to the height of the input, and W refers to the width of the input. Here, "input" refers to the input data.

[0063] Stage 0 consists of two layers. The first layer comprises three sequential operations: a convolutional neural network (Conv), batch normalization (BN), and an activation function. The activation function can be, for example, a rectified linear unit (ReLU). The Conv kernel size is 7×7, the number of kernels is 64, and the stride is 2 (represented by " / 2" in the diagram). The second layer is a max pooling layer with a 3×3 kernel and a stride of 2. The output shape of Stage 0 is (64, 56, 56), where 64 equals the number of kernels in the first convolutional layer of Stage 0, and 56 equals 224 / 2 / 2 (a stride of 2 halves the input size).

[0064] Stage 1 consists of one bottleneck layer 1 and two bottleneck layers 2; Stage 2 consists of one bottleneck layer 1 and two bottleneck layers 2; Stage 3 consists of one bottleneck layer 1 and five bottleneck layers 2; Stage 4 consists of one bottleneck layer 1 and two bottleneck layers 2. The output shape of Stage 1 is (256, 56, 56); the output shape of Stage 2 is (512, 28, 28); the output shape of Stage 3 is (1024, 14, 14); and the output shape of Stage 4 is (2048, 7, 7).

[0065] The number of input and output channels in bottleneck layer 1 is different. Bottleneck layer 1 has four parameters: C, W, C1, and S. C refers to the number of input channels, W refers to the height and width of the input, C1 refers to the number of features output by the convolutional layer (i.e., the number of output channels), and S refers to the stride in the convolutional layer. When S is 1, the input and output sizes are the same, indicating that no downsampling is performed. The right side of bottleneck layer 1 first passes through a convolutional layer (including Batch Normalization), changing its output channel count. Let its output be G(x). The left side of bottleneck layer 1 passes through three convolutional layers (including Batch Normalization and activation functions), letting its output be F(x). The right convolutional layer matches the number of output channels of the left side. Thus, the right side adds F(x) and G(x) and then passes it through the activation function to obtain the output of bottleneck layer 1.

[0066] Bottleneck layer 2 has the same number of input and output channels, but it lacks one convolutional layer on the right side compared to bottleneck layer 1. Bottleneck layer 2 has two parameters, C and W, where C refers to the number of input channels and W refers to the height and width of the input. Bottleneck layer 2 passes through three convolutional layers on the left (including batch normalization and activation functions). Let its output be F(x). Adding F(x) to x and then passing it through the activation function yields the output of bottleneck layer 2.

[0067] It should be noted that the multi-stream feature extractor provided in this application includes multiple feature extractors. Each feature extractor can be a feature extraction network used to preprocess each sample image. For example, each feature extractor can be stage 0 of ResNet50, or a combination of stage 0 and stage 1 of ResNet50, etc. In addition, each feature extractor can process sample images in batches.

[0068] Optionally, for each sample image, the computer device inputs the sample image into a feature extractor corresponding to the source of the sample image, based on the source of the sample image, to obtain the features extracted by that feature extractor. The features extracted by each feature extractor are then fused to obtain the sample fused features.

[0069] Specifically, feature fusion can be performed by concatenating the features extracted by each feature extractor or by superimposing the features extracted by each feature extractor.

[0070] It should be noted that before inputting multiple sample images from different sources into the multi-stream feature extractor, the computer device can normalize the multiple sample images and then input the normalized sample images into the multi-stream feature extractor. That is, the computer device normalizes the multiple sample images to obtain multiple normalized sample images; for each normalized sample image, the computer device inputs the normalized sample image into the feature extractor corresponding to the source of the normalized sample image, based on the source of the normalized sample image, to obtain the features extracted by that feature extractor.

[0071] In other words, extracting features from multiple sample images using the multi-stream feature extractor of the model to be trained can be achieved by extracting features from multiple normalized sample images using the multi-stream feature extractor of the model to be trained. Using normalized sample images to train the model to be trained can improve the training speed of the model.

[0072] In this embodiment, the computer device processes sample images from different sources through different feature extractors in the multi-stream feature extractor. This allows each feature extractor to focus on extracting high-quality features from the sample images of its corresponding source. This design ensures that the features extracted from sample images from various sources can achieve effective feature fusion while maintaining the maximum amount of information.

[0073] Step 206: Extract multi-scale features from the sample fusion features using the multi-scale network of the model to be trained, so as to obtain the sample target features of the target part.

[0074] Multi-scale networks are network structures capable of processing and fusing information at different scales (size, angle, etc.) to improve a model's adaptability and recognition accuracy to different sizes and levels of detail. A multi-scale network consists of multiple feature extraction layers at different scales. Each feature extraction layer can be a feature extraction network itself. For example, a multi-scale network may include stages 1, 2, and 3 in ResNet50, or stages 1, 2, 3, and 4, or stages 2 and 3, etc.

[0075] Multi-scale feature extraction is performed on the sample fusion features of the model to be trained using a multi-scale network to obtain the sample target features of the target part. This can be done by directly using the features output by the multi-scale network as the sample target features of the target part, or by further processing the features output by the multi-scale network to obtain the sample target features of the target part.

[0076] In an optional implementation, the computer device performs multi-scale feature extraction on the sample fusion features through a multi-scale network, and directly uses the features output by the smallest scale feature extraction layer in the multi-scale network as the sample target features of the target part.

[0077] In another optional implementation, the model to be trained further includes an attention network. The computer device performs multi-scale feature extraction on the sample fusion features through feature extraction layers of different scales to obtain feature extraction results of different scales. The attention network performs attention processing on the feature extraction results of different scales, and obtains the sample target features of the target part based on the attention processing results.

[0078] Obtaining sample target features of the target region based on the attention processing results can be achieved by directly using the attention processing results as sample target features of the target region, or by using a computer device to extract features from the attention processing results through a feature extraction layer of at least one scale in a multi-scale network to obtain sample target features of the target region.

[0079] It should be noted that the "feature extraction layers of different scales" in multi-scale feature extraction of sample fusion features through feature extraction layers of different scales is different from the "feature extraction layer of at least one scale" in computer equipment for feature extraction of attention processing results through feature extraction layers of at least one scale in a multi-scale network.

[0080] In this embodiment, the computer device extracts multi-scale features from the sample fusion features using a multi-scale network of the model to be trained. This enables the mining of multi-scale features from sample images from different sources, ensuring that the target features of the obtained target area can match the images captured by each image acquisition device. In other words, extracting multi-scale features from the sample fusion features using a multi-scale network helps maintain feature consistency even under the specific influence of image acquisition devices, such as resolution variations.

[0081] Step 208: Train the model to be trained based on the target features of the target area and the training labels corresponding to multiple sample images, and obtain the feature extraction model after training is completed.

[0082] The training labels can be stored features of target parts of sample objects in the database, or they can be the identity identifiers of the sample objects, i.e., the real identity identifiers of the sample objects. This database stores stored features of target parts of multiple different objects.

[0083] Optionally, based on the target features of the target area and the training labels corresponding to multiple sample images, a cross-entropy loss is constructed to obtain the target loss, and the model to be trained is trained according to the target loss.

[0084] In one alternative implementation, the cross-entropy loss can be constructed by a computer device based on the target features of the sample and the storage features of the sample object. Alternatively, the computer device can match the target features of the sample with the various storage features in the database, and construct the cross-entropy loss based on the identity identifiers of the matched storage features and the identity identifiers of the sample object.

[0085] In another alternative implementation, the computer device further constructs a triplet loss, where the triples in the construction of the triplet loss are the anchor point, a positive sample, and a negative sample. The triples are sample images from a batch of sample images input into the model to be trained. Specifically, positive samples are sample images belonging to the same sample object as the anchor point, and negative samples are sample images not belonging to the same sample object as the anchor point.

[0086] The triplet loss is constructed based on anchor points, positive samples, and negative samples, and the sum of the cross-entropy loss and the triplet loss is used as the target loss.

[0087] In one alternative implementation, the computer device determines the identity identifier that best matches the sample target features based on the sample target features, including: calculating the similarity between the sample target features and each stored feature in the database, and using the identity identifier of the object corresponding to the stored feature with the highest similarity as the identity identifier that best matches the sample target features.

[0088] The above-described model processing method acquires multiple sample images from different sources, including images captured by different image acquisition devices of the target area of ​​the sample object. Features from these multiple sample images are then extracted using a multi-stream feature extractor of the model to be trained. Different feature extractors are used to process sample images from different sources, allowing each extractor to focus on extracting high-quality features from its corresponding source. Based on the extracted features, sample fusion features are obtained. These fusion features are then used by a multi-scale network of the model to be trained to extract multi-scale features, resulting in the target features of the target area. This method extracts multi-scale features from sample images obtained from different sources (different devices), ensuring that these features match across different sources. This helps maintain consistency of target area features despite the specific influences (such as resolution) brought about by different device sources. Training the model using the target features extracted in this way and the training labels corresponding to the multiple sample images allows the model to learn key features unaffected by specific image acquisition devices, thereby improving the accuracy of image feature extraction and ultimately enhancing the model's generalization ability. When using the feature extraction model obtained after training for identity recognition, the feature extraction model can extract key features from the image to be processed that are not affected by specific image acquisition devices, thereby improving the accuracy of identity recognition.

[0089] In one embodiment, the sample images include first sample images and second sample images. Acquiring multiple sample images from different sources includes: acquiring multiple first sample images obtained by different image acquisition devices capturing images of the target parts of the sample object; wherein, one type of image acquisition device corresponds to one source; performing cross-device style conversion on the multiple first sample images to obtain multiple second sample images; wherein, one device-like style obtained by cross-device style conversion corresponds to one source.

[0090] In this context, one type of image acquisition device corresponds to one device style, while a cross-device style transfer corresponds to a device-like style. Device-like style refers to the device style transferred through deep learning networks. For example, using a deep learning network to transform an image taken by image acquisition device X into an image that looks like it was taken by image acquisition device Y.

[0091] Each first sample image is an image obtained by capturing the target part of the sample object using each image acquisition device. In other words, the source of the first sample image is determined according to the type of image acquisition device. The corresponding second sample image is an image obtained by performing cross-device style conversion on each first sample image. In other words, the source of the second sample image is determined according to the device style.

[0092] Optionally, the computer device acquires multiple first sample images of the target parts of the sample object from different publicly available datasets, obtained by different image acquisition devices. For each first sample image, at least one cross-device style transfer is performed to obtain at least one second sample image corresponding to the first sample image.

[0093] Specifically, for each first sample image, at least one cross-device style conversion is performed on the first sample image. That is, for each first sample image, the first sample image is converted from one device style to another device style, or the first sample image is converted from one device style to multiple different other device styles.

[0094] For example, two first sample images are obtained from two different public datasets, which are taken by image acquisition device X and image acquisition device Y, respectively; the first sample image taken by image acquisition device X is converted to the style of image acquisition device Y, and the second sample image taken by image acquisition device Y is converted to the style of image acquisition device X, thus obtaining two second sample images.

[0095] In this embodiment, the sample images include first sample images and second sample images. Multiple sample images from different sources are obtained by acquiring multiple first sample images by capturing the target parts of the sample object with different image acquisition devices, and by performing cross-device style conversion on the multiple first sample images to obtain multiple second sample images. This not only expands the training data, but also provides richer modal semantic features for the model to be trained, thereby effectively improving the adaptability and robustness of the feature extraction model in the face of device diversity.

[0096] In one embodiment, performing cross-device style transfer on multiple first sample images to obtain multiple second sample images includes: for each first sample image, inputting the first sample image into a preset network to obtain a second sample image output by the preset network; the second sample image has a different device style from the corresponding first sample image, and the preset network includes at least one of a generation network or a diffusion network.

[0097] The preset network is a deep learning network model, which is a trained network capable of realizing device style transfer of images. The preset network can be a generative network, a diffusion network, or a combination of generative and diffusion networks.

[0098] Both generative networks and diffusion networks (DNs) are trained networks that enable device style transfer of images.

[0099] Generative networks can be, for example, Cycle-Consistent Generative Adversarial Networks (CycleGAN). (See reference...) Figure 4 The CycleGAN architecture diagram shown below includes a generator: and and discriminator D X and D Y Here, X and Y represent the image styles obtained by image acquisition devices X and Y respectively from capturing images of the target parts of the sample object. Generator G can convert the image style captured by image acquisition device X into the image style captured by image acquisition device Y, and generator F can convert the image style captured by image acquisition device Y into the image style captured by image acquisition device X.

[0100] The training process of the generative network is as follows: The generative network to be trained includes the generator G, the generator F, and the discriminator D. X and the discriminator D to be trained Y Based on the adversarial loss function Loss GAN and cycle consistency loss function Loss cycle The sum is used to train the generative network to be trained.

[0101] The adversarial loss function is as follows:

[0102]

[0103]

[0104] The cycle consistency loss function is as follows:

[0105]

[0106] Where E represents the expected value, , Both represent a data distribution. express In sample y comes from Expectation under distribution express In sample x from Expectation under distribution express In sample x from Expectation under distribution express In sample y comes from Expected value when the distribution is as follows. This represents the similarity between sample x, which is generated by generator G to resemble a point x similar to Y, and the point x is then restored by generator F to its original form x. This indicates the similarity between sample y, which is generated by generator F to resemble the tip of y similar to X, and y, which is then restored by generator G, and the original sample y.

[0107] The diffusion network comprises a first sub-diffusion network and a second sub-diffusion network. The first sub-diffusion network converts the style of an image captured by image acquisition device X to the style of an image captured by image acquisition device Y. Conversely, the second sub-diffusion network converts the style of an image captured by image acquisition device Y to the style of an image captured by image acquisition device X. The training process of the diffusion network includes training the first sub-diffusion network and training the second sub-diffusion network.

[0108] The training process of the first sub-diffusion network is as follows: The first style training sample image captured by the image acquisition device X is input into the first sub-diffusion network to be trained, and the first style conversion image output by the first sub-diffusion network to be trained is obtained. Based on the difference between the first style conversion image output by the first sub-diffusion network to be trained and the second style training sample image captured by the image acquisition device Y, the first sub-diffusion network to be trained is trained.

[0109] The training process of the second sub-diffusion network is as follows: The second style training sample image captured by the image acquisition device Y is input into the second sub-diffusion network to be trained, and the second style conversion image output by the second sub-diffusion network to be trained is obtained. Based on the difference between the second style conversion image output by the second sub-diffusion network to be trained and the first style training sample image captured by the image acquisition device X, the second sub-diffusion network to be trained is trained.

[0110] Optionally, when the preset network is a generator network, for each sample image, the first sample image is input into the generator in the generator network corresponding to the source of the first sample image, and the second sample image output by the generator network is obtained.

[0111] When the preset network is a diffusion network, for each sample image, the first sample image is input into the sub-diffusion network corresponding to the source of the first sample image in the diffusion network to obtain the second sample image output by the diffusion network.

[0112] In this embodiment, for each first sample image, the second sample image output by the preset network is obtained by inputting the first sample image into the preset network. This method of obtaining the second sample image through the preset network is simple and fast.

[0113] In one embodiment, features of multiple sample images are extracted by a multi-stream feature extractor of the model to be trained, and sample fusion features are obtained based on the extracted features. This includes: for each sample image, inputting the sample image into one of the feature extractors in the multi-stream feature extractor of the model to be trained; each feature extractor performing feature extraction on its respective input sample image; and fusing the features extracted by each feature extractor to obtain sample fusion features.

[0114] Optionally, as can be seen from the above, multiple sample images can include only multiple first sample images, or multiple first sample images and multiple second sample images. The structure of the multi-stream feature extractor will also be different depending on the number of sample images.

[0115] Reference Figure 5 As shown, taking multiple sample images, including only multiple first sample images, as an example, the multi-stream feature extractor extracts features from multiple sample images, and the principle of obtaining sample fusion features based on the extracted features is as follows:

[0116] Each first sample image is input into its corresponding feature extractor in the multi-stream feature extractor; each feature extractor extracts features from its respective input first sample image; the features extracted by each feature extractor are fused to obtain the sample fusion feature.

[0117] For example, both the first sample image captured by image acquisition device X and the first sample image captured by image acquisition device Y are used as sample images in the source domain. These first sample images are then input into their respective feature extractors, each extracting features from its input first sample image. The features extracted by each feature extractor are then fused to obtain the sample fused features. The multi-stream feature extractor includes two feature extractors.

[0118] In other words, during the training process, the data input to the model to be trained is The obtained sample fusion features .in, This represents the first sample image captured by image acquisition device X. This represents the first sample image captured by image acquisition device Y. This represents the features extracted by the feature extractor corresponding to the first sample image captured by image acquisition device X. This represents the features extracted by the feature extractor corresponding to the first sample image captured by image acquisition device Y. express , The fusion features of the samples obtained after fusion.

[0119] Reference Figure 6As shown, taking multiple sample images, including multiple first sample images and multiple second sample images, as an example, the multi-stream feature extractor extracts features from multiple sample images, and the principle of obtaining sample fusion features based on the extracted features is as follows:

[0120] Each first sample image and each second sample image are input into their respective feature extractors in the multi-stream feature extractor; each feature extractor performs feature extraction on its respective input first sample image or second sample image; the features extracted by each feature extractor are fused to obtain the sample fusion feature.

[0121] For example, both the first sample image captured by image acquisition device X and the first sample image captured by image acquisition device Y are used as sample images in the source domain. The second sample images obtained by performing cross-device style transfer on each of these first sample images are used as sample images in the target domain. The first sample images captured by image acquisition device X, image acquisition device Y, the second sample images obtained by performing cross-device style transfer on the first sample image captured by image acquisition device X, and the second sample images obtained by performing cross-device style transfer on the first sample image captured by image acquisition device Y are respectively input into their corresponding feature extractors. Each feature extractor extracts features from its respective input first or second sample image. The features extracted by each feature extractor are then fused to obtain the sample fused features. The multi-stream feature extractor includes four feature extractors.

[0122] In other words, during the training process, the data input to the model to be trained is The obtained sample fusion features .in, This represents the first sample image captured by image acquisition device X. This represents the second sample image obtained after performing cross-device style transfer on the first sample image captured by image acquisition device X. This represents the first sample image captured by image acquisition device Y. This represents the second sample image obtained after performing cross-device style transfer on the first sample image captured by image acquisition device Y; This represents the features extracted by the feature extractor corresponding to the first sample image captured by image acquisition device X. This represents the features extracted by the feature extractor corresponding to the second sample image obtained after performing cross-device style transfer on the first sample image captured by image acquisition device X. This represents the features extracted by the feature extractor corresponding to the first sample image captured by image acquisition device Y. This represents the features extracted by the feature extractor corresponding to the second sample image obtained after performing cross-device style transfer on the first sample image captured by image acquisition device Y; express , , , The fusion features of the samples obtained after fusion.

[0123] In this embodiment, for each sample image, the computer device inputs the sample image into one of the feature extractors in the multi-stream feature extractor of the model to be trained; each feature extractor extracts features from its own input sample image; and the features extracted by each feature extractor are fused to obtain sample fusion features. This enables the features to be concatenated during training to encourage the model to use independent feature extractors to extract more diverse modality-specific features, rather than focusing on shared features, thereby improving the model's generalization ability.

[0124] In one embodiment, the multi-scale network includes multiple feature extraction layers of different scales, and the model to be trained also includes an attention network. The multi-scale feature extraction of the sample fusion features is performed through the multi-scale network of the model to be trained to obtain the sample target features of the target part, including: performing multi-scale feature extraction of the sample fusion features through feature extraction layers of different scales to obtain feature extraction results of different scales; and performing attention processing on the feature extraction results of different scales through the attention network to obtain the sample target features of the target part based on the attention processing results.

[0125] In this context, the feature extraction layers at different scales can be all the feature extraction layers in the multi-scale network, or they can be some of the feature extraction layers in the multi-scale network. For example, when the multi-scale network is stage1, stage2, stage3 and stage4 in ResNet50, multi-scale feature extraction of sample fusion features can be performed by using stage1, stage2 and stage3 to perform multi-scale feature extraction of sample fusion features, or it can be performed by using stage1, stage2, stage3 and stage4 to perform multi-scale feature extraction of sample fusion features.

[0126] Optionally, the computer device performs multi-scale feature extraction on the sample fusion features through feature extraction layers of different scales to obtain feature extraction results at different scales. Specifically, the first feature extraction layer extracts features from the sample fusion features to obtain the feature extraction result output by the first feature extraction layer; the next scale feature extraction layer extracts features from the feature extraction result output by the previous scale feature extraction layer to obtain the feature extraction result output by the next scale feature extraction layer, wherein the extraction scale of the previous scale feature extraction layer is larger than the extraction scale of the next scale feature extraction layer.

[0127] Taking feature extraction layers of different scales, including stage1, stage2 and stage3, as an example, stage1 extracts features from the sample fusion feature G0 to obtain the feature extraction result G1 output by stage1. Stage2 extracts features from the feature extraction result G1 output by stage1 to obtain the feature extraction result G2 output by stage2. Stage3 extracts features from the feature extraction result G2 output by stage2 to obtain the feature extraction result G3 output by stage3.

[0128] Taking feature extraction layers of different scales, including stage1, stage2, stage3 and stage4, as an example, based on the above embodiment of feature extraction layers of different scales including stage1, stage2 and stage3, stage4 performs feature extraction on the feature extraction result G3 output by stage3 to obtain the feature extraction result G4 output by stage4.

[0129] Attention networks are network structures that enable models to automatically focus on important parts of the input data. Attention networks can be multi-layered, allowing models to automatically learn the optimal relationships between features at different scales.

[0130] Optionally, the attention network includes N attention processing layers, where N is a positive integer greater than 1. Attention processing is performed on the feature extraction results at different scales through the attention network, including: the first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer to obtain the output of the first attention processing layer; for any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)th attention processing layer and the feature extraction results output by the (i-1)th feature extraction layer to obtain the output of the i-th attention processing layer; where i is a positive integer greater than 1 and less than or equal to N; the output of the Nth attention processing layer is used as the attention processing result.

[0131] Reference Figure 7 The architecture of the attention processing layer shown is as follows: the i-th attention processing layer performs attention processing on the output of the (i-1)-th attention processing layer and the feature extraction result of the (i-1)-th feature extraction layer to obtain the output of the i-th attention processing layer, including: processing the output F of the (i-1)-th attention processing layer. i-1 , and the feature extraction result G output by the (i-1)th feature extraction layer i-1 The input is fed into the i-th attention processing layer, and the i-th attention processing layer processes the output F of the (i-1)-th attention processing layer. i-1 and the feature extraction result G output by the (i-1)th feature extraction layeri-1 After adjustment, the output F of the (i-1)th attention processing layer is then processed. i-1 And the feature extraction result G output by the adjusted (i-1)th feature extraction layer. i-1 Perform attention processing to obtain the output F of the i-th attention processing layer. i .

[0132] Similarly, the first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer, resulting in the output of the first attention processing layer, which includes: the sample fusion features G0 and the feature extraction results G0 output by the (N-1)th feature extraction layer. N-1 The input is fed into the first attention processing layer, which processes the sample fusion feature G0 and the feature extraction result G from the (N-1)th feature extraction layer. N-1 After adjustment, the fused feature G0 of the adjusted sample and the feature extraction result G output from the adjusted (N-1)th feature extraction layer are performed. N-1 Attention processing is performed to obtain the output F1 of the first attention processing layer.

[0133] The adjustment process includes adjusting the number of channels and / or adjusting the feature size. Adjusting the number of channels can be achieved by adjusting the number of 1×1 convolutional layers, and adjusting the feature size can be achieved by adjusting the stride of the 1×1 convolutional layers. For example, the output F of the (i-1)th attention layer can be adjusted using a 1×1 convolutional layer. i-1 The feature extraction result G output by the (i-1)th feature extraction layer i-1 Sample fusion feature G0, and feature extraction result G output from the (N-1)th feature extraction layer. N-1 The number of channels was adjusted to 256, and the feature size was adjusted to (24, 12).

[0134] Reference Figure 7 As shown, the adjusted output F of the (i-1)th attention processing layer i-1 And the feature extraction result G output by the adjusted (i-1)th feature extraction layer. i-1 Perform attention processing to obtain the output F of the i-th attention processing layer. i This includes: the computer device's output F of the adjusted (i-1)th attention processing layer. i-1 And the feature extraction result G output by the adjusted (i-1)th feature extraction layer. i-1 Feature fusion is performed by adding features pixel by pixel. The fused features are then subjected to channel attention and spatial attention processing, and the number of channels is restored to obtain the output F of the i-th attention processing layer. i .

[0135] Similarly, the adjusted sample fusion feature G0 and the adjusted feature extraction result G from the (N-1)th feature extraction layer are used. N-1 Attention processing is performed to obtain the output F1 of the first attention processing layer, which includes: the fused feature G0 of the adjusted samples, and the feature extraction result G of the adjusted (N-1)th feature extraction layer. N-1 Feature fusion is performed by adding features pixel by pixel. The fused features are then subjected to channel attention and spatial attention processing, and the number of channels is restored to obtain the output F1 of the first attention processing layer.

[0136] Among these, feature fusion by adding pixels one by one can be weighted fusion, for example, F i-1 +0.1G i-1 Channel count restoration can also be achieved using 1×1 convolutional layers. For example, the output channel count of each attention layer can be restored to 1024 channels. Furthermore, the output F of the Nth attention layer can be restored... N As a result of attention processing.

[0137] In this embodiment, the first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer to obtain the output of the first attention processing layer. For any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)th attention processing layer and the feature extraction results output by the (i-1)th feature extraction layer to obtain the output of the i-th attention processing layer. The output of the Nth attention processing layer is used as the attention processing result. This enables the model to automatically learn the optimal association between features at different scales, has strong long-range dependency modeling capabilities, can effectively capture global context information, reduce information loss, and retain complementary information of features at different scales.

[0138] Optionally, obtaining sample target features of the target area based on the attention processing result includes: the computer device directly using the attention processing result as the sample target feature of the target area, or processing the attention processing result to obtain the sample target feature of the target area.

[0139] The processing of attention processing results can be performed by computer equipment to batch normalize the attention processing results to obtain sample target features of the target area, or by computer equipment to extract features from the attention processing results through a feature extraction layer of at least one scale to obtain sample target features of the target area.

[0140] In this embodiment, the computer device performs multi-scale feature extraction on the sample fusion features through feature extraction layers of different scales to obtain feature extraction results at different scales. Then, an attention network is used to process the feature extraction results at different scales. Based on the attention processing results, the sample target features of the target area are obtained. Since the attention network has the characteristic of enabling the model to automatically focus on the important parts of the input data, by processing the feature extraction results at different scales through the attention network, the important parts of the feature extraction results at different scales can be further obtained. Thus, the sample target features of the obtained target area are key features that are not affected by specific image acquisition devices, thereby further improving the generalization ability of the model.

[0141] In one embodiment, obtaining sample target features of the target region based on the attention processing result includes: extracting features from the attention processing result through a feature extraction layer of at least one scale to obtain sample target features of the target region.

[0142] Among them, at least one feature extraction layer includes a feature extraction layer of one scale, or multiple feature extraction layers of different scales.

[0143] In at least one feature extraction layer, the extraction scale of each feature extraction layer is smaller than that of other feature extraction layers in the multi-scale network. This is to enable further feature extraction from the attention processing results to obtain key features. The feature extraction layer at at least one scale can be stage 4 in ResNet50.

[0144] In this embodiment, the computer device extracts features from the attention processing results through a feature extraction layer of at least one scale, which can further obtain key features in the attention processing results, so that the sample target features of the obtained target parts are key features that are not affected by specific image acquisition devices, thereby further improving the generalization ability of the model.

[0145] In one embodiment, the training of the model to be trained is performed based on the sample target features of the target part and the training labels corresponding to multiple sample images, including: matching the sample target features with each stored feature in the database; and training the model to be trained based on the difference between the identity identifier of the matched stored feature and the identity identifier of the sample object.

[0146] The database stores the storage characteristics of target parts of multiple different objects.

[0147] Optionally, the computer device calculates the similarity between the sample target features and each stored feature in the database, and uses the identity identifier of the object corresponding to the stored feature with the highest similarity as the identity identifier with the highest matching degree with the sample target features. Based on the difference between the identity identifier with the highest matching degree with the sample target features and the identity identifier of the sample object, the model to be trained is trained.

[0148] In this embodiment, by matching the target features of the sample with the stored features in the database, and training the model to be trained based on the difference between the identity identifier of the matched stored features and the identity identifier of the sample object, the trained model can be applied to identity recognition scenarios.

[0149] In summary, the model to be trained has multiple architectures, including multi-stream feature extractor extraction and multi-scale network, or multi-stream feature extractor extraction, multi-scale network and attention network.

[0150] The multi-stream feature extractor includes multiple feature extractors. For example, each feature extractor can be stage 0 of ResNet50, or a combination of stage 0 and stage 1 of ResNet50. Furthermore, the number of feature extractors can be two, used to process the first sample image captured by image acquisition device X and the first sample image captured by image acquisition device Y, respectively; or the number of feature extractors can be four, used to process the first sample image captured by image acquisition device X, the second sample image obtained after cross-device style transfer of the first sample image captured by image acquisition device X, the first sample image captured by image acquisition device Y, and the second sample image obtained after cross-device style transfer of the first sample image captured by image acquisition device Y, respectively.

[0151] Multi-scale networks include multiple feature extraction layers of different scales. For example, a multi-scale network may include stage 1, stage 2 and stage 3 in ResNet50, or stage 1, stage 2, stage 3 and stage 4, or stage 2 and stage 3, etc.

[0152] An attention network consists of N attention processing layers, and the architecture of the attention processing layers is as follows: Figure 7 As shown, the attention network can be placed after the multi-scale network. The output of the Nth attention processing layer is the attention processing result, which is used as the sample target feature of the target region. Alternatively,

[0153] An attention network is positioned between two feature extraction layers in a multi-scale network. It extracts features from the attention processing results through at least one feature extraction layer of each scale, obtaining the target features of the target region. For example, stage 4 extracts features from the attention processing results to obtain the target features of the target region.

[0154] Preferred, select such as Figure 8 The architecture of the model to be trained shown is simple and has strong generalization ability.

[0155] The architecture of the model to be trained includes a multi-stream feature extractor, a multi-scale network, and an attention network.

[0156] Multi-stream feature extractor reference Figure 6 The multi-stream feature extractor shown comprises four feature extractors, each representing stage 0 of ResNet50. The multi-scale network includes four feature extraction layers at different scales, designated as stage 1, stage 2, stage 3, and stage 4 in ResNet50. The attention network consists of four attention processing layers, with the architecture of each layer as shown below. Figure 7 As shown, the attention network is set between stage 3 and stage 4.

[0157] During training, the first sample image captured by image acquisition device X, the second sample image obtained by cross-device style transformation of the first sample image captured by image acquisition device X, the first sample image captured by image acquisition device Y, and the second sample image obtained by cross-device style transformation of the first sample image captured by image acquisition device Y are respectively input into the corresponding stage0. Each stage0 performs feature extraction on its own input first or second sample image, and the features extracted by each stage0 are fused to obtain the sample fusion feature G0.

[0158] Stage 1 extracts features from the sample fusion feature G0, resulting in the feature extraction result G1. Stage 2 extracts features from the feature extraction result G1, resulting in the feature extraction result G2. Stage 3 extracts features from the feature extraction result G2, resulting in the feature extraction result G3.

[0159] Adjust the number of channels and / or feature size of sample fusion feature G0, feature extraction result G1, feature extraction result G2, and feature extraction result G3 to the preset number of channels and preset feature size. For example:

[0160] Before adjustment, the sample fusion feature G0 was (64, 96, 48). After adjusting the number of channels and feature size, the sample fusion feature G0 is (256, 24, 12).

[0161] Before adjustment, the feature extraction result G1 was (256, 96, 48). After adjusting the number of channels and feature size, the feature extraction result G1 was (256, 24, 12).

[0162] Before adjustment, the feature extraction result G2 was (512, 48, 24). After adjusting the number of channels and feature size, the feature extraction result G2 was (256, 24, 12).

[0163] Before adjustment, the feature extraction result G3 was (1024, 24, 12). After adjusting the number of channels, the feature extraction result G3 is (256, 24, 12).

[0164] The first attention processing layer applies attention to the adjusted sample fusion feature G0 and the adjusted feature extraction result G3, resulting in the output F1 of the first attention processing layer. Note: The output of each attention processing layer is the result after channel number restoration. Therefore, when the next attention processing layer applies attention to the output of the previous attention processing layer, the channel number of the output of the previous attention processing layer needs to be adjusted.

[0165] The second attention processing layer performs attention processing on the adjusted feature extraction result G1 and the adjusted output F1 of the first attention processing layer to obtain the output F2 of the second attention processing layer.

[0166] The third attention processing layer performs attention processing on the adjusted feature extraction result G2 and the adjusted output F2 of the second attention processing layer to obtain the output F3 of the third attention processing layer.

[0167] The fourth attention processing layer performs attention processing on the adjusted feature extraction result G3 and the adjusted output F3 of the third attention processing layer to obtain the output F4 of the fourth attention processing layer.

[0168] Stage 4 extracts features from the output F4 of the fourth attention processing layer to obtain the sample target features of the target area.

[0169] Based on the target features of the samples and the training labels corresponding to multiple sample images, a cross-entropy loss is constructed to obtain the target loss, and the model to be trained is trained according to the target loss.

[0170] After training, a feature extraction model is obtained, and then the feature extraction model is tested to evaluate its performance.

[0171] Obtain query and gallery sets from different public datasets. For example, the query set contains images of target parts of different test objects taken by image acquisition device X, and the gallery set contains images of target parts of different test objects taken by image acquisition device Y.

[0172] If cross-device style transfer images are not used during training, then during testing, for each image in the query set, the images in the query set are input into the corresponding feature extractor in the feature extraction model to obtain the first test target feature output by the feature extraction model. Each image in the gallery set is input into the corresponding feature extractor in the feature extraction model to obtain multiple second test target features output by the feature extraction model.

[0173] For each image in the query set, the similarity between the first test target feature and each second test target feature is calculated. Based on the similarity results, each image in the gallery set is sorted from high to low. It is then determined whether the images in the gallery set that match the images in the query set are among the top k images in the sorted results, thereby determining the accuracy of the feature extraction model prediction under different levels of attention.

[0174] If cross-device style-transformed images are used during training, then during testing, for each image in the query set, the images in the query set and the images obtained after cross-device style transformation are input into the corresponding feature extractor in the feature extraction model to obtain the first test target feature output by the feature extraction model. For each image in the gallery set, and the images obtained after cross-device style transformation are input into the corresponding feature extractor in the feature extraction model to obtain multiple second test target features output by the feature extraction model.

[0175] For each image in the query set, the similarity between the first test target feature and each second test target feature is calculated. Based on the similarity results, each image in the gallery set is sorted from high to low. It is then determined whether the images in the gallery set that match the images in the query set are among the top k images in the sorted results, thereby determining the accuracy of the feature extraction model prediction under different levels of attention.

[0176] This application conducted experimental evaluations on two publicly available cross-device palmprint datasets, focusing on feature extraction models trained using images that incorporated cross-device style transfer during training. For each dataset, the training and test sets were randomly split in a 1:1 ratio, and the average of multiple experimental results was taken to ensure the reliability of the results. The experimental results are shown in Table 1.

[0177] Table 1

[0178]

[0179] For each dataset, the training and test sets were randomly split in a 1:3 ratio, and the average of multiple experimental results was taken to ensure the reliability of the results. The experimental results are shown in Table 2.

[0180] Table 2

[0181]

[0182] Here, Rank-k represents the prediction accuracy of the feature extraction model when the images in the gallery set that match the images in the query set are among the top k in the ranking results. For example, Rank-10 represents the prediction accuracy of the feature extraction model when the images in the gallery set that match the images in the query set are among the top k in the ranking results.

[0183] Fold i (where i is a positive integer, greater than or equal to 0) refers to the (i+1)th fold validation performed during cross-validation. For example, Fold0 refers to the first fold validation performed during cross-validation. Cross-validation is a technique for evaluating the generalization ability of a machine learning model. The process involves dividing the dataset into several parts (folds), and then using one part as the test set and the rest as the training set to train and validate the model.

[0184] MAP (Mean Average Precision) measures the average precision of images in the gallery set across the entire retrieval database and is one of the key metrics for evaluating the quality of feature extraction models.

[0185] In one embodiment, this application provides a model processing method, which comprises the most detailed steps for implementing this application:

[0186] Step 301: Acquire multiple first sample images obtained by different image acquisition devices capturing images of the target parts of the sample object; wherein, one type of image acquisition device corresponds to one source; for each first sample image, input the first sample image into a preset network to obtain a second sample image output by the preset network; the device style of the second sample image is different from that of the corresponding first sample image, wherein, a device style obtained by cross-device style conversion corresponds to one source; the preset network includes at least one of a generation network or a diffusion network.

[0187] Step 302: For each sample image, input the sample image into one of the feature extractors in the multi-stream feature extractor of the model to be trained; each feature extractor extracts features from its respective input sample image; and the features extracted by each feature extractor are fused to obtain the sample fusion feature.

[0188] Step 303: The model to be trained includes a multi-scale network. The sample fusion features are extracted by feature extraction layers of different scales in the multi-scale network to obtain feature extraction results of different scales.

[0189] Step 304: The model to be trained also includes an attention network, which includes N attention processing layers, where N is a positive integer greater than 1. The first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer to obtain the output of the first attention processing layer. For any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)th attention processing layer and the feature extraction results output by the (i-1)th feature extraction layer to obtain the output of the i-th attention processing layer. Here, i is a positive integer greater than 1 and less than or equal to N. The output of the Nth attention processing layer is taken as the attention processing result.

[0190] Step 305: Extract features from the attention processing results through a feature extraction layer of at least one scale to obtain the sample target features of the target region.

[0191] Step 306: Match the target features of the sample with the stored features in the database; based on the difference between the identity identifier of the matched stored features and the identity identifier of the sample object, train the model to be trained, and obtain the feature extraction model after training.

[0192] In one embodiment, such as Figure 9 As shown, an image recognition method is provided, which is applied to a computer device (specifically, the computer device may be...) Figure 1 Taking terminal 102 or server 104 as an example, the following steps are included:

[0193] Step 902: Obtain the image to be processed; the image to be processed includes the image obtained by taking pictures of the target part of the target object.

[0194] The target part of the target object refers to the part that can be used to uniquely identify the target object, such as the palm print, face, or fingerprint of the target object.

[0195] Optionally, the image to be processed is related to the training data used in the training of the feature extraction model. If the sample images used in the training of the feature extraction model are only images taken by different image acquisition devices, then the computer device acquires the image to be processed obtained by the image acquisition device.

[0196] If, during the training of the feature extraction model, the sample images used include not only images taken by different image acquisition devices, but also images obtained by cross-device style conversion of images taken by image acquisition devices, then the computer device acquires the first image taken by the image acquisition device and performs cross-device style conversion on the first image to obtain the second image, thereby obtaining an image to be processed that includes the first image and the second image.

[0197] The computer device can perform cross-style conversion on the first image by inputting the first image into a preset network to obtain a second image output by the preset network; the second image has a different device style from the corresponding first image, and the preset network includes at least one of a generation network or a diffusion network.

[0198] Step 904: The feature extraction model is invoked to process the image to be processed to obtain the target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Different feature extractors are used to process sample images from different sources, and the multi-scale network is used to extract multi-scale features from the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor.

[0199] Based on the above-described model processing method embodiments, it can be seen that the feature extraction model includes a multi-stream feature extractor and a multi-scale network, or it includes a multi-stream feature extractor, a multi-scale network, and an attention network. Different feature extraction models involve different steps in processing the image to be processed.

[0200] Optionally, the computer device invokes a feature extraction model, inputting the image to be processed into the feature extractor in the feature extraction model corresponding to the source of the image to be processed, so as to obtain the target features output by the feature extraction model.

[0201] When the feature extraction model includes a multi-stream feature extractor and a multi-scale network, the multi-scale network performs multi-scale feature extraction on the features extracted by the feature extractor and directly outputs the target features.

[0202] When a feature extraction model includes a multi-stream feature extractor, a multi-scale network, and an attention network, the computer device performs multi-scale feature extraction on the features extracted by the feature extractor through feature extraction layers of different scales in the multi-scale network, and obtains feature extraction results of different scales; then, the attention network performs attention processing on the feature extraction results of different scales, and the target features are obtained based on the attention processing results.

[0203] Obtaining target features based on attention processing results can be achieved by directly using the attention processing results as sample target features of the target area, or by using a computer device to extract features from the attention processing results through a feature extraction layer of at least one scale in a multi-scale network to obtain sample target features of the target area.

[0204] Step 906: Perform identity recognition based on target features to obtain the identity recognition result of the target object.

[0205] Among them, the identification result of the target object is the result used to uniquely identify the target object. For example, the identification result of the target object can be the identity identifier of the target object.

[0206] In one optional implementation, the similarity between the target feature and each stored feature in the database is calculated, and the identity identifier of the object corresponding to the stored feature with the highest similarity is used as the identity recognition result of the target object.

[0207] The aforementioned image recognition method acquires an image to be processed, which includes images captured of the target parts of a target object. A feature extraction model is then used to process the image to obtain target features. This feature extraction model comprises a multi-stream feature extractor and a multi-scale network, trained from multiple sample images from different sources. Different feature extractors process sample images from different sources, while the multi-scale network performs multi-scale feature extraction on the sample fusion features, which are obtained by fusing features extracted by each feature extractor. Then, identity recognition is performed based on the target features to obtain the target object's identity. This process, where different feature extractors in the multi-stream feature extractor extract features from sample images captured by different image acquisition devices, followed by multi-scale feature extraction, and then training the model based on these target features, allows the model to learn key features unaffected by specific image acquisition devices. This improves the model's accuracy in image feature extraction and, consequently, its generalization ability. Therefore, when the trained feature extraction model is applied, it can extract key features from the image to be processed that are unaffected by specific image acquisition devices, thus improving the accuracy of identity recognition.

[0208] In one embodiment, a feature extraction model is invoked to process the image to be processed to obtain target features, including: determining the source of the image to be processed; using the image to be processed as input to a feature extractor corresponding to the source, so as to process the image to be processed through the feature extraction model to obtain target features.

[0209] Here, the source of the image to be processed refers to its origin. For example, what type of device it was taken with, or whether it is a device-style image obtained through cross-device style conversion.

[0210] Optionally, the source of the image to be processed can be determined by the type of image acquisition device used to capture the image. If the image to be processed is a cross-device style transfer image, the source of the image to be processed can be determined by whether it is a device-like style transfer image.

[0211] Specifically, when a computer device acquires an image to be processed, it also acquires the image parameters of the image to be processed, including the source of the image. In this way, the computer device can determine the source of the image to be processed at the same time as acquiring the image.

[0212] Optionally, the image to be processed is used as input to a feature extractor corresponding to the source, so that the image to be processed is processed by a feature extraction model to obtain target features. This includes: the computer device inputs the image to be processed into the feature extractor corresponding to the source to obtain the features extracted by the feature extractor corresponding to the source of the image to be processed; and a multi-scale network performs multi-scale feature extraction on the features extracted by the feature extractor to obtain target features.

[0213] In this embodiment, the source of the image to be processed is determined, and then the image to be processed is used as input to the feature extractor corresponding to the source. The feature extraction model processes the image to obtain the target features. Since each feature extractor only extracts features from sample images corresponding to its own source during the training process, the trained feature extraction model has the ability to extract high-quality features from images from different sources. Therefore, inputting the image to be processed into the feature extractor corresponding to the source ensures that the obtained target features are key features.

[0214] In one embodiment, the image to be processed includes a first image captured by an image acquisition device and a second image obtained by performing cross-device style transfer on the first image; wherein the first image corresponds to a first source and the second image corresponds to a second source; the image to be processed is used as input to a feature extractor corresponding to the source, so as to process the image to be processed through a feature extraction model to obtain target features, including: extracting features of the first image through a feature extractor corresponding to the first source in a multi-stream feature extractor; extracting features of the second image through a feature extractor corresponding to the second source in a multi-stream feature extractor; obtaining fused features based on the extracted features of the first image and the features of the second image; and performing multi-scale feature extraction on the fused features through a multi-scale network to obtain target features of the target region.

[0215] The first source is the source corresponding to the type of image acquisition device that captured the first image; the second source is the source corresponding to the device-like style obtained through cross-device style conversion.

[0216] Optionally, the computer device inputs the first image into the feature extractor corresponding to the first source in the multi-stream feature extractor, obtaining the features of the first image extracted by the feature extractor corresponding to the first source. The computer device inputs the second image into the feature extractor corresponding to the second source in the multi-stream feature extractor, obtaining the features of the second image extracted by the feature extractor corresponding to the second source. The computer device concatenates the extracted features of the first image and the features of the second image to obtain fused features. Multi-scale feature extraction is performed on the fused features through a multi-scale network to obtain the target features of the target region.

[0217] The multi-scale network includes multiple feature extraction layers of different scales, and the model to be trained also includes an attention network.

[0218] Optionally, the computer device performs multi-scale feature extraction on the fused features through feature extraction layers of different scales to obtain feature extraction results of different scales; and performs attention processing on the feature extraction results of different scales through an attention network to obtain sample target features of the target part based on the attention processing results.

[0219] In one possible implementation, the computer device performs multi-scale feature extraction on the fused features through feature extraction layers of different scales to obtain feature extraction results of different scales. Specifically, the first feature extraction layer extracts features from the fused features to obtain the feature extraction result output by the first feature extraction layer; the next scale feature extraction layer extracts features from the feature extraction result output by the previous scale feature extraction layer to obtain the feature extraction result output by the next scale feature extraction layer, wherein the extraction scale of the previous scale feature extraction layer is larger than the extraction scale of the next scale feature extraction layer.

[0220] In one possible implementation, the attention network includes N attention processing layers, where N is a positive integer greater than 1. Attention processing is performed on the feature extraction results at different scales through the attention network, including: the first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer to obtain the output of the first attention processing layer; for any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)th attention processing layer and the feature extraction results output by the (i-1)th feature extraction layer to obtain the output of the i-th attention processing layer; where i is a positive integer greater than 1 and less than or equal to N; and the output of the Nth attention processing layer is used as the attention processing result.

[0221] In one possible implementation, obtaining sample target features of the target region based on the attention processing result includes: the computer device directly using the attention processing result as the target feature of the target region, or processing the attention processing result to obtain the sample target feature of the target region.

[0222] The processing of attention processing results can be performed by computer equipment to batch normalize the attention processing results to obtain sample target features of the target area, or by computer equipment to extract features from the attention processing results through a feature extraction layer of at least one scale to obtain sample target features of the target area.

[0223] In this embodiment, by inputting the first image and the second image after cross-device style conversion of the first image into the corresponding feature extractors, the features of the two different images are preserved, thereby increasing the accuracy of identity recognition.

[0224] In one embodiment, identity recognition is performed based on target features to obtain the identity recognition result of the target object, including: comparing the target features with each stored feature in the database; if a stored feature matching the target features is obtained based on the comparison result, the identity identifier corresponding to the stored feature is used as the identity identifier of the target object.

[0225] The database stores the stored features of target parts of different objects. The comparison result refers to the degree of difference between the target feature and each stored feature. The degree of difference between the target feature and each stored feature can be characterized by, for example, the similarity between the target feature and each stored feature, or by the distance between the target feature and each stored feature.

[0226] In one optional implementation, the similarity between the target feature and each stored feature in the database is calculated, and the identity identifier of the object corresponding to the stored feature with the highest similarity is used as the identity identifier of the target object.

[0227] In another optional implementation, the distance between the target feature and each stored feature in the database is calculated, and the identity identifier of the object corresponding to the stored feature with the smallest distance is used as the identity identifier of the target object.

[0228] In this embodiment, by comparing the target feature with each stored feature in the database, if a stored feature that matches the target feature is obtained based on the comparison result, the identity identifier corresponding to the stored feature is used as the identity identifier of the target object, thereby achieving the purpose of identity recognition while ensuring personal information security.

[0229] In one embodiment, the method further includes: if the identity recognition result indicates that the identity of the target object has been verified, then controlling the IoT device to perform a preset operation; or, if the identity recognition result includes an identity identifier, then performing a resource transfer operation based on the identity identifier.

[0230] The IoT devices can be, in a passenger scenario, devices that scan target areas (such as facial recognition, hand recognition, fingerprint recognition, and iris recognition devices), or in a door-opening scenario, access control devices that scan target areas (facial recognition, hand recognition, fingerprint recognition, and iris recognition devices). Preset operations refer to device control operations; for example, in a passenger scenario, this could be a gate opening operation, or in a door-opening scenario, it could be a door opening or closing operation.

[0231] In a vehicle-riding scenario, the computer device acquires a first palmprint image of the target object captured by a hand-scanning device. The computer device then performs cross-device style conversion on the first palmprint image through a pre-defined network to obtain a second palmprint image in a style similar to that of a mobile phone camera. A feature extraction model is then used to process both the first and second palmprint images to obtain target features. Based on these features, identity verification is performed to obtain the target object's identity verification result. If the identity verification result indicates that the target object's identity has been verified, the computer device sends an opening request to the gate control device, causing the gate control device to execute the opening operation.

[0232] In the door opening scenario, the computer device acquires a first palmprint image of the target object captured by the hand-scanning access control device. The computer device then performs cross-device style conversion on the first palmprint image through a preset network to obtain a second palmprint image in a style similar to that of a mobile phone camera. A feature extraction model is then used to process both the first and second palmprint images to obtain target features. Based on these features, identity recognition is performed to obtain the target object's identity verification result. If the identity recognition result indicates that the target object's identity has been verified, the computer device sends an opening request to the door opening control device, causing the door opening control device to execute the opening operation.

[0233] In a payment scenario, a computer device captures a first palmprint image of the target object. The computer device then performs cross-device style transformation on the first palmprint image through a pre-defined network to obtain a second palmprint image in a style similar to that of a mobile phone camera. A feature extraction model is then used to process both the first and second palmprint images to obtain target features. Based on these features, identity recognition is performed, resulting in an identity verification result containing an identity identifier. The computer device then performs a resource transfer operation based on the identity identifier.

[0234] In this embodiment, IoT devices are controlled to perform preset operations or resource transfer operations through identity recognition, thereby realizing intelligent living.

[0235] In one embodiment, this application provides an image recognition method, which includes the following detailed steps for implementing this application:

[0236] Step 401: Obtain a first image by taking a picture of the target part of the target object; perform cross-device style conversion on the first image to obtain at least one second image, and obtain an image to be processed including the first image and the second image.

[0237] Step 402: The first image corresponds to the first source and the second image corresponds to the second source. The features of the first image are extracted using the feature extractor corresponding to the first source in the multi-stream feature extractor. The features of the second image are extracted using the feature extractor corresponding to the second source in the multi-stream feature extractor. The fused features are obtained based on the extracted features of the first image and the features of the second image. The fused features are then subjected to multi-scale feature extraction using a multi-scale network to obtain the target features of the target region.

[0238] Step 403: Compare the target feature with each stored feature in the database. If a stored feature that matches the target feature is obtained based on the comparison result, then the identity identifier corresponding to the stored feature is used as the identity identifier of the target object.

[0239] Step 404: If the identity recognition result indicates that the identity of the target object has been verified, then control the IoT device to perform a preset operation; or, if the identity recognition result includes an identity identifier, then perform a resource transfer operation based on the identity identifier.

[0240] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0241] Based on the same inventive concept, this application also provides a model processing apparatus for implementing the model processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more model processing apparatus embodiments provided below can be found in the limitations of the model processing method described above, and will not be repeated here.

[0242] In one embodiment, such as Figure 10 As shown, a model processing device 1000 is provided, comprising: an acquisition module 1001, a first feature extraction module 1002, a second feature extraction module 1003, and a training module 1004, wherein:

[0243] The acquisition module 1001 is used to acquire multiple sample images from different sources; the multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object;

[0244] The first feature extraction module 1002 is used to extract features from multiple sample images through the multi-stream feature extractor of the model to be trained, and to obtain sample fusion features based on the extracted features; different feature extractors are used to process sample images from different sources;

[0245] The second feature extraction module 1003 is used to extract multi-scale features from the sample fusion features through the multi-scale network of the model to be trained, so as to obtain the sample target features of the target part.

[0246] The training module 1004 is used to train the model to be trained based on the sample target features of the target part and the training labels corresponding to the multiple sample images, and obtain the feature extraction model after the training is completed.

[0247] In one embodiment, the acquisition module 1001 is specifically used to acquire multiple first sample images obtained by different image acquisition devices capturing the target part of the sample object; wherein, one type of image acquisition device corresponds to one source; and multiple second sample images are obtained by performing cross-device style conversion on the multiple first sample images; wherein, one device-like style obtained by cross-device style conversion corresponds to one source.

[0248] In one embodiment, the acquisition module 1001 is specifically used to input the first sample image into a preset network for each first sample image to obtain a second sample image output by the preset network; the second sample image has a different device style from the corresponding first sample image, and the preset network includes at least one of a generation network or a diffusion network.

[0249] In one embodiment, the first feature extraction module 1002 is specifically used to input the sample image to one of the feature extractors in the multi-stream feature extractor of the model to be trained for each sample image; each feature extractor performs feature extraction on its respective input sample image; and the features extracted by each feature extractor are fused to obtain the sample fusion feature.

[0250] In one embodiment, the multi-scale network includes multiple feature extraction layers of different scales, and the model to be trained also includes an attention network; the second feature extraction module 1003 is specifically used to perform multi-scale feature extraction on the sample fusion features through the feature extraction layers of different scales to obtain feature extraction results of different scales; and to perform attention processing on the feature extraction results of different scales through the attention network to obtain the sample target features of the target part based on the attention processing results.

[0251] In one embodiment, the attention network includes N attention processing layers, where N is a positive integer greater than 1; the second feature extraction module 1003 is specifically used to perform attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer by the first attention processing layer of the attention network to obtain the output of the first attention processing layer; for any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)th attention processing layer and the feature extraction results output by the (i-1)th feature extraction layer to obtain the output of the i-th attention processing layer; where i is a positive integer greater than 1 and less than or equal to N; and the output of the Nth attention processing layer is used as the attention processing result.

[0252] In one embodiment, the second feature extraction module 1003 is specifically used to extract features from the attention processing results through a feature extraction layer of at least one scale to obtain sample target features of the target part.

[0253] In one embodiment, the training module 1004 matches the sample target features with various stored features in the database; and trains the model to be trained based on the difference between the identity identifier of the matched stored features and the identity identifier of the sample object.

[0254] Based on the same inventive concept, this application also provides an image recognition device for implementing the image recognition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more image recognition device embodiments provided below can be found in the limitations of the image recognition method described above, and will not be repeated here.

[0255] In one embodiment, such as Figure 11 As shown, an image recognition device is provided. The image recognition device 1100 includes: an acquisition module 1101, a processing module 1102, and a recognition module 1103, wherein:

[0256] The acquisition module 1101 is used to acquire the image to be processed; the image to be processed includes an image obtained by capturing the target part of the target object;

[0257] The processing module 1102 is used to call the feature extraction model to process the image to be processed and obtain the target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Among them, different feature extractors are used to process sample images from different sources, and the multi-scale network is used to perform multi-scale feature extraction on the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor.

[0258] The identification module 1103 is used to identify the target based on the target features and obtain the identification result of the target object.

[0259] In one embodiment, the image to be processed includes a first image and a second image. The acquisition module 1101 is specifically used to acquire the first image obtained by taking a picture of the target part of the target object; and to perform cross-device style conversion on the first image to obtain at least one second image.

[0260] In one embodiment, the processing module 1102 is specifically used to determine the source of the image to be processed; and to use the image to be processed as input to a feature extractor corresponding to the source, so as to process the image to be processed through a feature extraction model to obtain target features.

[0261] In one embodiment, the image to be processed includes a first image captured by an image acquisition device and a second image obtained by performing cross-device style conversion on the first image; wherein the first image corresponds to a first source and the second image corresponds to a second source; the processing module 1102 is specifically used to extract features of the first image through the feature extractor corresponding to the first source in the multi-stream feature extractor; extract features of the second image through the feature extractor corresponding to the second source in the multi-stream feature extractor; obtain fused features based on the extracted features of the first image and the features of the second image; and perform multi-scale feature extraction on the fused features through a multi-scale network to obtain target features of the target region.

[0262] In one embodiment, the identification module 1103 is specifically used to compare the target feature with each stored feature in the database. If a stored feature that matches the target feature is obtained according to the comparison result, the identity identifier corresponding to the stored feature is used as the identity identifier of the target object.

[0263] In one embodiment, the device further includes an execution module, configured to control the IoT device to perform a preset operation if the identity recognition result indicates that the identity of the target object has been verified; or, if the identity recognition result includes an identity identifier, to perform a resource transfer operation based on the identity identifier.

[0264] Each module in the aforementioned model processing device and image recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0265] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as sample images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a model processing method.

[0266] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image recognition method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0267] Those skilled in the art will understand that Figure 12 and Figure 13The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0268] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any one of the methods in the above-described module processing method embodiments or the image recognition method embodiments.

[0269] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in any one of the above-described module processing method embodiments or image recognition method embodiments.

[0270] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any one of the methods in the above-described module processing method embodiments or the image recognition method embodiments.

[0271] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0272] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model processing method, characterized in that, The method includes: Acquire multiple sample images from different sources; the multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object; Features are extracted from the multiple sample images using a multi-stream feature extractor of the model to be trained, and sample fusion features are obtained based on the extracted features; different feature extractors are used to process sample images from different sources; The multi-scale feature extraction of the sample fusion features is performed by the multi-scale network of the model to be trained to obtain the sample target features of the target part; The model to be trained is trained based on the target features of the target region and the training labels corresponding to the multiple sample images, and a feature extraction model is obtained after training.

2. The method according to claim 1, characterized in that, The sample images include a first sample image and a second sample image. The acquisition of multiple sample images from different sources includes: Multiple first sample images are obtained by capturing the target part of the sample object with different image acquisition devices; wherein, one type of image acquisition device corresponds to one source. Multiple first sample images are subjected to cross-device style transfer to obtain multiple second sample images; among them, a device-like style obtained by cross-device style transfer corresponds to a source.

3. The method according to claim 1, characterized in that, The process of extracting features from the multiple sample images using a multi-stream feature extractor of the model to be trained, and obtaining sample fusion features based on the extracted features, includes: For each sample image, the sample image is input to one of the feature extractors in the multi-stream feature extractor of the model to be trained; each feature extractor extracts features from its respective input sample image; The sample fusion feature is obtained by fusing the features extracted by each feature extractor.

4. The method according to claim 1, characterized in that, The multi-scale network includes multiple feature extraction layers of different scales, and the model to be trained also includes an attention network. The step of extracting multi-scale features from the sample fusion features using the multi-scale network of the model to be trained, in order to obtain the sample target features of the target region, includes: Multi-scale feature extraction is performed on the sample fusion features by feature extraction layers of different scales to obtain feature extraction results of different scales; The attention network is used to process the feature extraction results at different scales, and the sample target features of the target region are obtained based on the attention processing results.

5. The method according to claim 4, characterized in that, The attention network includes N attention processing layers, where N is a positive integer greater than 1; The attention processing of the feature extraction results at different scales through the attention network includes: The first attention processing layer of the attention network performs attention processing on the sample fusion features and the feature extraction results output by the (N-1)th feature extraction layer to obtain the output of the first attention processing layer. For any i-th attention processing layer other than the first one, the i-th attention processing layer performs attention processing on the output of the (i-1)-th attention processing layer and the feature extraction result of the (i-1)-th feature extraction layer to obtain the output of the i-th attention processing layer; where i is a positive integer greater than 1 and less than or equal to N; The output of the Nth attention processing layer is used as the attention processing result.

6. The method according to any one of claims 1-5, characterized in that, The step of training the model to be trained based on the sample target features of the target region and the training labels corresponding to the multiple sample images includes: The target features of the sample are matched with the stored features in the database; The model to be trained is trained based on the difference between the identity identifier of the matching stored features and the identity identifier of the sample object.

7. An image recognition method, characterized in that, The method includes: Acquire the image to be processed; the image to be processed includes an image obtained by capturing a target part of the target object; The feature extraction model is invoked to process the image to be processed to obtain target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Different feature extractors are used to process sample images from different sources, and the multi-scale network is used to perform multi-scale feature extraction on the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor. Based on the target features, identity recognition is performed to obtain the identity recognition result of the target object.

8. The method according to claim 7, characterized in that, The image to be processed includes a first image and a second image, and the process of obtaining the image to be processed includes: Acquire the first image obtained by photographing the target part of the target object; Perform cross-device style transfer on the first image to obtain at least one second image.

9. The method according to claim 7, characterized in that, The process of calling the feature extraction model to process the image to be processed to obtain target features includes: Determine the source of the image to be processed; The image to be processed is used as input to a feature extractor corresponding to the source, so that the image to be processed is processed by a feature extraction model to obtain target features.

10. The method according to claim 9, characterized in that, The image to be processed includes a first image captured by an image acquisition device, and a second image obtained by performing cross-device style conversion on the first image; wherein the first image corresponds to a first source and the second image corresponds to a second source; The step of using the image to be processed as input to a feature extractor corresponding to the source, and processing the image to be processed through a feature extraction model to obtain target features, includes: The features of the first image are extracted using the feature extractor corresponding to the first source in the multi-stream feature extractor. Features of the second image are extracted using the feature extractor corresponding to the second source in the multi-stream feature extractor. The fused features are obtained based on the extracted features of the first image and the features of the second image; The multi-scale network is used to extract multi-scale features from the fused features to obtain the target features of the target region.

11. A model processing device, characterized in that, The device includes: The acquisition module is used to acquire multiple sample images from different sources; the multiple sample images include images obtained by different image acquisition devices capturing the target parts of the sample object; The first feature extraction module is used to extract features from the multiple sample images through the multi-stream feature extractor of the model to be trained, and to obtain sample fusion features based on the extracted features; different feature extractors are used to process sample images from different sources; The second feature extraction module is used to perform multi-scale feature extraction on the sample fusion features through the multi-scale network of the model to be trained, so as to obtain the sample target features of the target part; The training module is used to obtain a feature extraction model after training based on the sample target features of the target region and the training labels corresponding to the multiple sample images.

12. An image recognition device, characterized in that, The device includes: An acquisition module is used to acquire an image to be processed; the image to be processed includes an image obtained by capturing a target part of a target object; The processing module is used to call the feature extraction model to process the image to be processed and obtain the target features. The feature extraction model includes a multi-stream feature extractor and a multi-scale network, which are trained from multiple sample images from different sources. Different feature extractors are used to process sample images from different sources, and the multi-scale network is used to perform multi-scale feature extraction on the sample fusion features. The sample fusion features are obtained by fusing the features extracted by each feature extractor. The identification module is used to perform identity recognition based on the target features and obtain the identity recognition result of the target object.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6 or 7 to 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6 or 7 to 10.

15. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6 or 7 to 10.

Images