A model determination method and related apparatus

Abstract

The application discloses a model determination method and related device, and aims at object type identification of an object in an image. A first image sample feature of an image sample can be determined through an initial encoder, and a second image sample feature identifying different identification environments is generated based on the first image sample feature. Both the two image sample features can obtain corresponding texture images through an initial decoder, and an initial classifier is used to determine identification results corresponding to the image sample and the cross-scene image sample respectively according to the image sample features and the corresponding texture images, and an initial identification model is trained based on an identification label and an identification loss function generated based on the identification result to obtain an identification model. The identification model trained in this way has good generalization ability for cross-scene identification and various attack type identification, and effectively improves the model identification precision.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a model determination method and related apparatus. Background Technology

[0002] A recognition model is a model that uses images to identify object types. By training an initial recognition model with a certain number of image samples, a recognition model with a certain recognition capability can be obtained. This recognition model can be applied to many aspects of production and daily life.

[0003] Because recognition models encounter various recognition scenarios and attack types in practical applications, they may even encounter new recognition scenarios and attack types that were never encountered during training. For example, when the recognition model is trained in an indoor scene, the recognition scenario encountered in actual application may be an outdoor scene. Or, when the recognition model is recognizing a live subject, the attack type during training may be to disguise a real live subject by taking a photo, while the attack type encountered in actual application may be to disguise a real live subject by replaying a video. In other words, recognition models will encounter a wide variety of recognition scenarios and attack types in practical applications.

[0004] However, the recognition models trained in related technologies struggle to achieve adequate generalization when performing object type recognition across scenarios and attack types, thus preventing the models from being widely applied. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a model determination method and related apparatus. During the training process, it not only enhances the recognition model's ability to identify the same object in different recognition environments, but also enables the model to learn more accurate recognition capabilities during training to effectively defend against various attack types, thereby obtaining a recognition model that achieves good generalization ability for cross-scene recognition and recognition of various attack types.

[0006] The embodiments of this application disclose the following technical solutions: On the one hand, embodiments of this application provide a model determination method, the method comprising: The initial encoder of the initial recognition model determines the first image sample features of the image sample. The image sample has an object type identification label that identifies the object in the image sample. The initial recognition model also includes an initial decoder and an initial classifier. The second image sample features are generated based on the features of the first image sample. The cross-scene image sample corresponding to the second image sample features has a different object type recognition environment than the image sample. The cross-scene image sample and the image sample have the same recognition label. Based on the features of the first image sample and the features of the second image sample, the first texture image corresponding to the image sample and the second texture image corresponding to the cross-scene image sample are obtained by the initial decoder; Based on the features of the first image sample and the first texture image, the first type of recognition result is determined by an initial classifier; based on the features of the second image sample and the second texture image, the second category of recognition result is determined by an initial classifier. Based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, a recognition loss function is generated. An initial recognition model is trained by recognizing a loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

[0007] In another aspect, embodiments of this application provide a model determination apparatus, which includes a first determination unit, a first generation unit, an acquisition unit, a second determination unit, a second generation unit, and a training unit. The first determining unit is used to determine the first image sample features of the image sample through the initial encoder of the initial recognition model. The image sample has an object label that identifies the object type in the image sample. The initial recognition model also includes an initial decoder and an initial classifier. The first generation unit is used to generate second image sample features based on the first image sample features. The cross-scene image sample corresponding to the second image sample features has a different object type recognition environment than the image sample. The cross-scene image sample and the image sample have the same recognition label. The acquisition unit is used to obtain, through an initial decoder, a first texture image corresponding to an image sample and a second texture image corresponding to a cross-scene image sample based on the features of the first image sample and the features of the second image sample. The second determining unit is used to determine a first type recognition result based on the features of the first image sample and the first texture image using an initial classifier; and to determine a second category recognition result based on the features of the second image sample and the second texture image using an initial classifier. The second generation unit is used to generate a recognition loss function based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, respectively. The training unit is used to train an initial recognition model using a recognition loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

[0008] In another aspect, embodiments of this application provide a computer device, which includes a processor and a memory: The memory is used to store computer programs and transfer them to the processor; The processor is used to execute the methods described above according to a computer program.

[0009] On another front, embodiments of this application provide a computer-readable storage medium for storing a computer program that performs the methods described above.

[0010] In another aspect, embodiments of this application provide a computer program product including a computer program, which, when run on a computer device, causes the computer device to perform the methods described above.

[0011] As can be seen from the above technical solution, for object type recognition in images, the initial recognition model to be trained includes an initial encoder, an initial decoder, and an initial classifier. The initial encoder determines the first image sample features of the image sample. To improve the model's ability to recognize the same object in different object type recognition environments, a second image sample feature is generated based on the first image sample feature to identify different recognition environments. The initial decoder can obtain corresponding texture images for both image sample features. Based on the image sample features and the corresponding texture images, the initial classifier determines the recognition results for the corresponding image sample and the cross-scene image sample, respectively. The initial recognition model is then trained based on the recognition labels and the recognition loss function generated from the recognition results to obtain the recognition model. Because the cross-scene transformation of image sample features allows the initial recognition model to learn how to recognize the same object in different object type recognition environments during training, and because this cross-scene transformation avoids encoding additional samples, it does not substantially affect the model's training efficiency. In addition to relying on image sample features, texture images were further introduced when performing object type recognition. This allowed for a richer representation of the image from both the feature dimension and the texture dimension, enabling the initial recognition model to learn more accurate recognition capabilities during training and cope with more diverse recognition attack methods. The recognition model trained in this way achieved good generalization ability in cross-scene recognition and various attack types when performing object recognition, effectively improving the model's recognition accuracy. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram illustrating a model-determined scenario provided in an embodiment of this application; Figure 2A flowchart illustrating a model determination method provided in this application embodiment; Figure 3 A schematic diagram illustrating cross-scene transformation of image sample features provided in an embodiment of this application; Figure 4 A schematic diagram of an attention mechanism provided in an embodiment of this application; Figure 5 A schematic diagram illustrating a detail-enhancing attention mechanism provided in an embodiment of this application; Figure 6 This is a schematic diagram illustrating a model determination method provided in an embodiment of this application; Figure 7 A test schematic diagram of a recognition model provided in an embodiment of this application; Figure 8 A schematic diagram of an apparatus for a model determination method provided in an embodiment of this application; Figure 9 A structural diagram of a terminal device provided in an embodiment of this application; Figure 10 This is a structural diagram of a server provided in an embodiment of this application. Detailed Implementation

[0014] The embodiments of this application will now be described with reference to the accompanying drawings.

[0015] Facial recognition models can be applied to various aspects of production and daily life. For example, a liveness detection model can identify whether a captured facial image comes from a real, living person, thus verifying the user's authenticity. This model helps to defend against fake face attacks, effectively ensuring the security of the facial recognition system. In other words, liveness detection models can be widely used as a crucial component of facial recognition systems in production and daily life. For instance, during remote bank account opening, the server uses a facial recognition system to verify the account holder's identity, which utilizes a liveness detection model. When the application front-end acquires an image containing the user's face through a camera... The application frontend transmits the image containing the user's face to the backend and calls the liveness detection model for liveness identification. If the image is confirmed to be live, subsequent identity verification will proceed; otherwise, verification will fail. In the process of facial payment using a facial recognition system, the liveness detection model can improve payment security, defending against transaction attacks that could harm businesses or individuals, thus ensuring the security of facial payments. When using a facial recognition system to ensure community security, after directly acquiring the facial image at the frontend, the system can feed the image into a pre-packaged liveness detection model for direct judgment.

[0016] Since the application value of a recognition model is determined by its generalization performance, the higher the generalization performance, the more application scenarios the model can be applied to, and the higher its application value. Therefore, to ensure the application value of a recognition model, it is necessary to ensure its generalization performance. Related technologies mainly focus on improving generalization performance through model training in two aspects. One aspect focuses on improving the generalization performance of the recognition model in various recognition scenarios, which may include unknown recognition scenarios (unknown domains). Unknown domains include characteristics such as different scenes, lighting, and the accuracy of the acquisition equipment compared to the training image samples. This aspect mainly includes techniques for image classification domain generalization tasks, using strategies such as domain adversarial training or meta-learning to allow the model to learn domain-independent features, enabling the model to achieve better generalization performance in unknown domains. For example, for liveness detection models, related technologies include Single-Side Domain Generalization for Face. The Anti-Spoofing (SSDG) algorithm uses a domain discriminator to distinguish live face features extracted by the feature extractor from different domains. When a live face feature extracted by the feature extractor can be easily identified as a live face by the domain discriminator, the feature is considered a cross-domain independent feature, and therefore may also have good recognition ability in unknown domains. On the other hand, it focuses on improving the detection ability of the recognition model against a variety of attack types, especially against novel attack types. For example, taking a liveness detection model as an example, attack types include photos, videos, face swaps, masks, and various other attacks that can provide fake faces. Novel attack types include those not seen in the training image samples. This aspect mainly improves the model's generalization performance against novel attack types by employing zero-shot learning or anomaly detection strategies, aiming to enable the model to learn attack cues that can generalize to novel attack types. For example, for liveness detection models, deep tree networks (Deep Tree Learning for Zero-shot) are a relevant technology. The FaceAnti-Spoofing (DTN) algorithm employs a zero-shot learning method. Known attack types are semantically divided into several clusters, and binary classification liveness detection is performed within each cluster. When faced with a new attack type, the image of the new attack type will be routed to the most similar cluster for binary classification, thereby improving the generalization ability of the liveness detection model in the face of new attack types.

[0017] However, in the actual application of object type recognition, the recognition model trained in the relevant technologies may not only face one of the recognition scenarios and new attack types, but may also face them simultaneously. For example, a liveness recognition model trained using photo attacks in indoor scenarios may encounter video replay attacks in outdoor scenarios in actual applications. That is, the recognition model will face multiple recognition scenarios and attack types, and may even encounter unknown recognition scenarios and new attack types at the same time. In this case, the model trained in the relevant technologies may not be able to achieve the appropriate generalization requirements.

[0018] To this end, this application provides a model determination method and related apparatus. During the training process, the recognition model not only improves its ability to recognize the same object in different recognition environments, but also learns more accurate recognition capabilities to effectively defend against various attack types, thereby obtaining a recognition model that achieves good generalization ability for cross-scene recognition and recognition of various attack types.

[0019] The model determination method provided in this application can be implemented using computer equipment, which can be a terminal device or a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this connection.

[0020] The model determination method provided in this application relates to artificial intelligence (AI) technology. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0021] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0022] The model determination method provided in this application mainly relates to computer vision technology in artificial intelligence. Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technology typically includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, and other technologies. It also includes common biometric recognition technologies such as face recognition and fingerprint recognition. The model determination method provided in this application can be used to determine the recognition models used in biometric recognition technologies such as image recognition, face recognition, and fingerprint recognition.

[0023] Figure 1 This is a schematic diagram of a model-determined scenario provided in an embodiment of this application, wherein the aforementioned computer device is a server.

[0024] A recognition model is a model that identifies object types through images. A recognition model can be obtained by training and optimizing an initial recognition model. In practical applications, recognition models will encounter various recognition scenarios and attack types, and may even encounter unknown recognition scenarios and new attack types that were not trained during the training process. When performing object type recognition, the recognition models trained in related technologies are difficult to achieve appropriate generalization requirements when facing cross-scene recognition and cross-attack type recognition, thereby reducing the application value of the recognition model.

[0025] Therefore, in the application scenario of this embodiment, the initial recognition model to be trained will include an initial encoder E, an initial decoder D, and an initial classifier C, such as... Figure 1As shown, the initial encoder can obtain image features that reflect image information in the feature dimension from the image samples, the initial decoder can generate a texture image that reflects image information in the texture dimension based on the image features, and the initial classifier can perform comprehensive recognition of the image based on the image features and the texture image.

[0026] Specifically, the server can generate first image sample features corresponding to image samples through an initial encoder. To improve the model's recognition ability in different recognition scenarios, the server generates second image sample features based on the first image sample features. The first and second image sample features identify different recognition environments but share the same recognition label. For both image sample features, the initial decoder can obtain corresponding texture images. Based on the image sample features and their corresponding texture images, the initial classifier determines the corresponding recognition results. The initial recognition model is then trained using a recognition loss function generated based on the recognition results and recognition labels to obtain the final recognition model. By generating second image sample features that identify different recognition environments but share the same recognition label using the first image sample features, and using both image sample features for training the initial recognition model, the model's ability to recognize the same object in different recognition environments can be improved by expanding the object type recognition environments encountered by the model.

[0027] Furthermore, the server directly performs cross-scene transformation on image features, avoiding the extra computational cost of first performing cross-scene transformation on image samples to obtain corresponding cross-scene image samples, and then using these cross-scene image samples to determine the features of the corresponding second image sample, thus ensuring the training efficiency of the model.

[0028] Furthermore, when the server performs object type identification through the initial classifier, it not only considers the features of the image samples but also introduces texture images. In other words, the initial classifier classifies the samples comprehensively from the feature dimension and the texture dimension of the image during the training process, enabling the initial recognition model to learn more accurate recognition capabilities during training. This, in turn, allows the obtained recognition model to cope with more diverse recognition attack methods.

[0029] The recognition model trained in this embodiment demonstrates good generalization ability and recognition accuracy for cross-scene recognition and recognition of various attack types when performing object recognition.

[0030] Figure 2 This application provides a flowchart of a model determination method, which can be executed by a computer device. In this embodiment, the computer device is a server as an example. The method includes: S201. The server determines the first image sample features of the image sample through the initial encoder of the initial recognition model. The image sample has an object recognition label that identifies the object type in the image sample. The initial recognition model also includes an initial decoder and an initial classifier.

[0031] A recognition model is a model that uses images to identify the type of an object. In practical applications, recognition models include various recognition models such as liveness detection models for identifying whether a currently acquired face image comes from a real living person, and vehicle recognition models for identifying whether a currently acquired vehicle image comes from a specific vehicle.

[0032] An initial recognition model refers to an initial model that can be used to train an image sample to obtain a recognition model, i.e., a recognition model that has not completed model training. The initial recognition model to be trained includes an initial encoder, an initial decoder, and an initial classifier. The initial encoder, initial decoder, and initial classifier refer to the encoder, decoder, and classifier that need to be trained and optimized, respectively. The encoder can obtain image features that can reflect image information in the feature dimension from the image. The decoder can generate a texture image that can reflect image information in the texture dimension based on the image features. The classifier can recognize the image based on the image information in the feature dimension and the texture dimension.

[0033] Image samples refer to images that can be used as training samples for a model. When the recognition model is a liveness detection model, the image samples can be various images including different faces. In order to achieve supervised learning training of the initial recognition model, the image samples include recognition labels. The recognition labels can identify the true value of the object type in the corresponding image sample. For example, when the recognition model is used to identify liveness, the recognition task of the face image sample is to identify whether the object in the sample is live. When the object type of the face in the face image sample is a real live, the corresponding recognition label content is live. Or, when the object type of the face in the face image sample is not a live, the corresponding recognition label content is not live. That is, the recognition label can reflect the true object type of the object in the image sample.

[0034] Image features refer to a series of quantifiable information reflected in an image. Image features can be used to characterize the content in an image or content related to the recognition task. In this embodiment, the first image sample feature refers to the image feature corresponding to the image sample.

[0035] In this embodiment, the server determines the first image sample feature corresponding to the image sample through the initial encoder. The first image sample feature can represent a series of information included in the image sample in the feature dimension, thereby realizing the preliminary analysis and processing of the image sample before recognition.

[0036] S202. The server generates second image sample features based on the features of the first image sample. The cross-scene image sample corresponding to the second image sample features has a different object type recognition environment than the image sample. The cross-scene image sample and the image sample have the same recognition label.

[0037] The object type recognition environment of an image refers to the content reflected in the image other than the object to be identified. For example, the object type recognition environment can include the background, lighting, and the accuracy of the acquisition device. Because the recognition model is affected by the object type recognition environment of the image, for instance, when an image sample is acquired in a well-lit environment, the recognition model trained on that image sample will have good recognition ability for images in well-lit environments. However, if the object type recognition environment is poorly lit, even if the image contains the same object as the image sample, the recognition model may not be able to accurately identify it. Therefore, it is necessary to improve the recognition model's ability to recognize the object type of the same object in different object type recognition environments.

[0038] In this embodiment, a second image sample feature can be generated based on the first image sample feature. This second image sample feature corresponds to a cross-scene image sample. A cross-scene image sample refers to a training sample that has a different object type recognition environment than the image sample but has the same recognition label. Specifically, the object type recognition environment of the cross-scene image sample is different from that of the image sample. For example, the image sample may be an indoor scene while the cross-scene image sample may be an outdoor scene. The recognition label of the cross-scene image sample is the same as that of the image sample, indicating that the objects included in both are of the same object type and are the same object. For example, when the recognition model is a liveness detection model, if the object type of the face in the image sample is live, the object type of the face in the cross-scene image sample will also be live and will be the same face. By obtaining a cross-scene image sample that includes the same object as the image sample, and using both the cross-scene image sample and the image sample for training the initial recognition model in subsequent steps, the initial recognition model is better able to recognize invariant features from the same object during subsequent training, thereby improving the model's ability to recognize the same object in different object type recognition environments.

[0039] It should be noted that in this embodiment, the server directly generates the second image sample features based on the first image sample features determined in S201, rather than first performing cross-scene transformation on the image samples to obtain the corresponding cross-scene image samples, and then determining the corresponding second image sample features based on the cross-scene image samples. This is because performing additional encoding processing on the cross-scene image samples through the initial encoder would increase the computational load during model training, thereby affecting the training efficiency of the model. Furthermore, since the first image sample features can reflect the information in the image samples more fully, the second image sample features obtained through the first image sample features can retain more image features from the original image samples.

[0040] The second image sample feature is generated by using the first image sample feature of the image sample. Since the cross-scene image sample and the image sample corresponding to the second image sample feature have different object type recognition environments, the object type recognition environments encountered during model training can be expanded, thereby improving the model's ability to recognize the same object in different object type recognition environments.

[0041] In one possible implementation, taking the j-th image sample as an example from a batch of M image samples trained on the initial recognition model, in step S202, the server generates second image sample features based on the features of the first image sample, specifically including: S11. The server determines the feature similarity parameters between the features of the j-th image sample and the remaining M-1 image samples in the M image samples; S12. The server identifies image samples with feature similarity parameters lower than the similarity threshold among M-1 image samples as differential samples; S13. The server generates second image sample features for the j-th image sample based on the first image sample features of the differential sample and the first image sample features of the j-th image sample.

[0042] The first image sample feature refers to the image feature corresponding to the image sample, and the second image sample feature refers to the image feature corresponding to the cross-scene image sample. Since the image sample and the cross-scene image sample have different object type recognition environments, the second image sample feature generated based on the first image sample feature needs to be significantly different from the first image sample feature.

[0043] To address this, for the j-th image sample, the server can first determine the feature similarity parameters between the first image sample features corresponding to the remaining M-1 image samples out of the M image samples and the first image sample corresponding to the j-th image sample. The feature similarity parameter represents the degree of similarity between different image samples in the dimension of image features. The higher the similarity of the first image sample features of different image samples, the larger the value of the corresponding feature similarity parameter. This feature similarity parameter can be cosine similarity. For example, the first image sample features of the same batch of image samples can be pooled into a sample feature vector v, and then the cosine similarity between the same batch of image samples can be determined using the following formula:

[0044] Where S represents the cosine similarity among the same batch of image samples. Let v represent the normalization function, and v represent the sample feature vector v determined by pooling the features of the first image sample in the same batch of image samples. Denotes the transpose of v. Let v denote the L2 norm of v.

[0045] The server can identify image samples with feature similarity parameters below a similarity threshold from M-1 image samples as dissimilar samples. Dissimilar samples are those that are significantly different from the j-th image sample. Since the features of the second image sample generated from the features of the first image sample need to be significantly different from the features of the first image sample, the aforementioned dissimilar samples can serve as the basis for generating the features of the second image sample. For example, when the feature similarity parameter is cosine similarity, the dissimilar samples can be determined using the following formula:

[0046] in, This indicates that the k-th image sample among M image samples is the first image sample feature corresponding to the difference sample of the j-th image sample. Indicates an indicator function, This represents the cosine similarity between the j-th image sample and the k-th image sample. This represents the cosine similarity between the j-th image sample and all M-1 image samples. This indicates finding the intermediate value of α.

[0047] This formula allows us to determine the similarity threshold as the median value of the cosine similarity between the j-th image sample and the M-1 image samples. Image samples among the M-1 image samples whose cosine similarity is lower than this median value are then identified as dissimilar samples. Figure 3 As shown, image samples that have a large difference from the j-th image sample are identified as difference samples.

[0048] After identifying the dissimilar samples, because cross-scene image samples and image samples have different object type recognition environments but the same recognition labels, the features of the second image sample, while needing to differ significantly from the features of the first image sample regarding the object type recognition environment, must have a high similarity in features regarding the recognition labels. Therefore, the server cannot directly generate the second image sample features based on the first image sample features of the dissimilar samples. Instead, it generates the second image sample features for the j-th image sample based on the first image sample features corresponding to both the dissimilar samples and the j-th image sample.

[0049] Furthermore, to further enhance the diversity of features in the second image samples, after identifying the dissimilar samples, a certain number of samples can be randomly discarded, for example, such as... Figure 3 As shown, half of the image samples can be randomly discarded, and then a second image sample feature for the j-th image sample can be generated based on the remaining difference samples and the j-th image sample.

[0050] The server first identifies the difference samples that differ significantly from the image samples. Then, based on the first image sample features corresponding to the difference samples and the image samples respectively, it generates second image sample features for the image samples. This results in cross-scene image samples that have different object type recognition environments from the image samples, thus extending the image samples to object type recognition environments.

[0051] In the process of generating the second image sample features, it is necessary to ensure that the cross-scene image samples and image samples corresponding to the second image sample features have the same recognition label. Therefore, in one possible implementation, in S13, the server generates the second image sample features for the j-th image sample based on the first image sample features of the differential samples and the first image sample features of the j-th image sample. Specifically, this includes: S21. The server generates initial second image sample features based on the first image sample features of the differential samples and the first image sample features of the j-th image sample. S22. The server uses the first image sample feature of the j-th image sample as a fusion constraint, and fuses the first image sample feature of the j-th image sample with the initial second image sample feature to generate the second image sample feature for the j-th image sample.

[0052] The server can first generate initial second image sample features for the j-th image sample based on the first image sample features corresponding to the difference samples and the j-th image sample, respectively. These initial second image sample features represent image features with different feature information than the first image sample features. These initial second image sample features carry the feature information of the first image sample features of the difference samples. Since the initial second image sample features are not equivalent to the second image sample features, and are essentially intermediate features used to generate the second image sample features, it is not necessary to guarantee that the initial second image sample features and the first image sample features represent the same object. For example, the initial second image sample features for the j-th image sample can be obtained according to the following formula:

[0053]

[0054] Where H represents the first image sample feature of the dissimilar samples for the j-th image sample, and S represents the cosine similarity among the image samples in the same batch. This represents the element-wise product of α and β. Indicates the discard function, The first image sample features represent the remaining differential samples after discarding. express Let V be the norm of the first image sample, and let V represent the first image sample feature of the j-th image sample. Identify the initial second image sample features of the j-th image sample.

[0055] In this formula, in order to further enhance the diversity of the second image sample features, not all the differential samples are used directly. Instead, the remaining differential samples after random discarding are obtained based on the differential samples. Then, the first image sample features of the remaining differential samples are regularized using a first norm to ensure that the sum of each row is 1. The initial second image sample features of the j-th image sample are determined by combining the first image sample features of the j-th image sample.

[0056] After generating initial second image sample features with significantly different feature information from the first image sample features, in order to ensure that the first and second image sample features represent the same object, the server can use the first image sample features of the j-th image sample as a fusion constraint during the fusion process of the first image sample features of the j-th image sample and the initial second image sample features. This indicates that the first image sample features of the j-th image sample and the initial second image sample features are not fused arbitrarily, but rather the original first image sample features are used as a constraint to ensure that the obtained second image sample features and the first image sample features represent the same object, that is, the first image sample features correspond to... The cross-scene image samples corresponding to the features of the first image sample and the second image sample have the same recognition label. For example, when the recognition model recognizes a live subject, using the features of the first image sample as a fusion constraint can ensure that the cross-scene image sample corresponding to the obtained second image sample features and the image sample depict the same face. This avoids errors when using the recognition label of the first image sample features as the corresponding ground truth value in subsequent steps, thus ensuring that the initial recognition model can be directly trained and optimized based on the second image sample features and recognition label in subsequent steps. Furthermore, it helps the model better recognize invariant features from the same object during subsequent training. Figure 3 As shown, the Exact Feature Distribution Mixing (EFDMix) technique can be used to fuse the feature information of the first image sample features and the initial second image sample features, thereby ensuring that the feature information of the first image sample features with respect to the recognition label remains unchanged, so as to obtain the corresponding second image sample features. At the same time, the Adaptive Instance Normalization (AdaIN) algorithm can also be used to fuse the first image sample features and the initial second image sample features.

[0057] First, an initial second image sample feature with different feature information from the first image sample feature is directly generated. Then, in the process of fusing the first image sample feature and the initial image sample feature, the original first image sample feature is used as a fusion constraint so that the obtained second image sample feature and the first image sample feature represent the same object. That is, the cross-scene image sample and the image sample corresponding to the second image sample feature have the same recognition label. This avoids errors when the recognition label of the first image sample feature is used as the corresponding real value in subsequent steps. This ensures that the initial recognition model can be directly trained and optimized based on the second image sample feature and the recognition label in subsequent steps.

[0058] S203. The server obtains the first texture image corresponding to the image sample and the second texture image corresponding to the cross-scene image sample through the initial decoder based on the features of the first image sample and the features of the second image sample.

[0059] A texture image is an image that reflects the texture of an image. For example, texture images include Local Binary Pattern (LBP) images, as well as color LBP images, LBP images in the HSV color space, or LBP images in the YCbCr color space, and other LBP variants. Texture is an image feature that reflects the distribution attributes of pixels in an image. It can be represented by the gray-level distribution of pixels and their surrounding spatial neighborhoods. The texture of an image is usually locally irregular but macroscopically regular. By acquiring the texture image of an image, a richer expression of the image can be achieved in the texture dimension. In this embodiment, the first texture image refers to the texture image corresponding to the image sample, and the second texture image refers to the texture image corresponding to the cross-scene image sample.

[0060] In this embodiment, the initial decoder can obtain the first texture image corresponding to the image sample based on the features of the first image sample. The features of the first image sample refer to a series of information reflected in the feature dimension of the sample image, while the first texture image refers to the visual information in the image reflected in the texture dimension of the sample image. That is, in this embodiment, in addition to performing feature dimension analysis and processing on the image sample, texture dimension analysis and processing are also performed. In other words, before recognizing the image sample, the server performs analysis and processing on the image sample in both feature and texture dimensions.

[0061] Furthermore, in this embodiment, the initial decoder can also obtain the second texture image corresponding to the cross-scene image sample based on the features of the second image sample. The features of the second image sample refer to a series of information reflected in the feature dimension of the cross-scene sample image, while the second texture image refers to the visual information in the image reflected in the texture dimension of the cross-scene sample image. That is, in this embodiment, in addition to obtaining the information of the cross-scene image sample in the feature dimension, the information in the texture dimension is also obtained, so that the cross-scene image sample obtains the corresponding information in both the feature and texture dimensions before recognition.

[0062] S204. The server determines the first type recognition result based on the features of the first image sample and the first texture image using an initial classifier; and determines the second category recognition result based on the features of the second image sample and the second texture image using an initial classifier.

[0063] Type recognition result refers to the predicted object type of the object in the image after the image is recognized by the model. The predicted result can be a probability value. For example, when the recognition model is a liveness detection model, the type recognition result obtained when the face image is recognized by the liveness detection model can be that there is a probability of 0.7 that it is a live object. In this embodiment, the first type recognition result refers to the type recognition result corresponding to the image sample, and the second type recognition result refers to the type recognition result corresponding to the cross-scene image sample.

[0064] In the training process, the recognition models in related technologies usually determine the corresponding type recognition result based only on the image sample features. This results in the recognition model being unable to recognize information in the image, and consequently, the recognition model trained in related technologies is unable to achieve the appropriate generalization requirement when facing cross-attack type recognition.

[0065] Since the recognition model may encounter various attack types in practical applications, including novel attack types not trained during the training process, in order to learn more accurate recognition capabilities during the training of the initial recognition model, in this embodiment, the server determines the first type recognition result corresponding to the image sample based on information from two dimensions: the features of the first image sample and the first texture image. In other words, in this embodiment, the initial classifier does not obtain the type recognition result corresponding to the image sample solely based on the features of the first image sample, but rather comprehensively obtains the corresponding first type recognition result by combining the features of the first image sample and the first texture image. That is, during the training process, the model classifies the image sample comprehensively from the feature dimension and the texture dimension of the image, thereby enabling the initial recognition model to more accurately and comprehensively identify and classify the information in the image sample during the training process.

[0066] Furthermore, in this embodiment, the second type of recognition result corresponding to the cross-scene image sample is determined by the initial classifier based on the information in the two dimensions of the second image sample features and the second texture image. In other words, the initial classifier also obtains the corresponding second type of recognition result by comprehensively considering the second image sample features and the second texture image. That is, the model also classifies the cross-scene image samples comprehensively from the feature dimension and the texture dimension of the image during the training process, so that the initial recognition model can more accurately and comprehensively identify and classify the information in the cross-scene image samples during the training process.

[0067] Based on the initial recognition model, which includes an initial encoder, an initial decoder, and an initial classifier, in one possible implementation, the initial recognition model further includes an initial feature embedding module. According to the features of the first image sample and the first texture image, in step S204, the server determines the first type of recognition result through the initial classifier, specifically including: The server determines the corresponding first embedding feature based on the features of the first image sample through the initial feature embedding module; The server determines the first type recognition result based on the first embedded feature and the first texture image using an initial classifier.

[0068] The initial feature embedding module refers to the feature embedding module that needs to be trained and optimized. The feature embedding module can perform secondary extraction of image features to obtain more accurate embedded features. Embedded features refer to features that can more accurately reflect the content information of the image.

[0069] To further improve the quality of the data input to the initial classifier, the server can determine the corresponding first embedding feature based on the features of the first image sample through the initial feature embedding module. That is, the image sample features are processed twice by the initial feature embedding module to obtain embedding features that can more accurately reflect the image content information.

[0070] Based on the first embedded feature obtained through the initial feature embedding module, the first embedded feature and the first texture image can be input into the initial classifier so that the initial classifier can perform classification and recognition. In this embodiment, the first embedded feature, which is more accurate than the first image sample feature, is input into the initial classifier, which is beneficial for the initial classifier to obtain a more accurate type recognition result.

[0071] Furthermore, in this embodiment, the initial feature embedding module can obtain the corresponding second embedding feature based on the features of the second image sample, and the second embedding feature and the second texture image can be input together into the initial classifier to determine the second type recognition result corresponding to the cross-scene image sample.

[0072] Based on the image sample features acquired by the server, the initial feature embedding module performs secondary analysis on the image sample features to obtain embedded features that more accurately reflect the image content information, thereby improving the data quality input to the initial classifier and helping the initial classifier to obtain more accurate type recognition results.

[0073] S205. The server generates a recognition loss function based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, respectively.

[0074] The first type of recognition result refers to the predicted object type of the objects included in the image sample after the image sample is recognized by the initial recognition model. The second type of recognition result refers to the predicted object type of the objects included in the cross-scene image sample after the cross-scene image sample is recognized by the initial recognition model. The recognition label refers to the label that identifies the true value of the object type of the objects in the image sample and the cross-scene image sample. The server can obtain a recognition loss function that reflects the training and optimization direction of the initial recognition model by the difference between the type recognition result and the recognition label of the initial recognition model. For example, for an initial recognition model with two classification results, the recognition loss function can be obtained by the following formula:

[0075] in, This refers to the recognition loss function, where y refers to the recognition label. This refers to either the first type of recognition result or the second type of recognition result. The recognition loss function of the initial recognition model with two classification results can be obtained through the above formula. For example, when the recognition model recognizes a live object, the object in the image sample is a real live object, and the corresponding recognition label value is 1. If the probability value of the first type of recognition result of the image sample is 0.8, the recognition loss function corresponding to the recognition label and the first type of recognition result can be calculated through the above formula.

[0076] It should be noted that since the second image sample features of cross-scene image samples are generated based on the first image sample features, the server can apply a certain weight to the loss corresponding to the first type of recognition result when generating the recognition loss function based on the differences between the first type of recognition result corresponding to the image sample and the second type of recognition result corresponding to the cross-scene image sample and the recognition label. For example, a weight of 0.1 can be applied to the loss corresponding to the first type of recognition result so that the recognition loss function still uses the actually acquired image samples as the main training samples.

[0077] S206. The server trains an initial recognition model using a recognition loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

[0078] After obtaining the recognition loss function in S205, since the recognition loss function can reflect the training and optimization direction of the initial recognition model, the server can train the initial recognition model according to the recognition loss function, thereby obtaining an applicable recognition model.

[0079] Since this recognition loss function is derived by combining the results of the first and second types of recognition, which correspond to the type recognition results of image samples and cross-scene image samples, respectively, the recognition model trained using this loss function achieves good generalization ability in cross-scene recognition. Furthermore, because the aforementioned type recognition results are obtained by combining the feature dimension and the texture dimension of the image, the recognition model trained using this loss function has more accurate and comprehensive recognition capabilities, and can cope with diverse recognition attack methods. In summary, this recognition model achieves good generalization ability for cross-scene recognition and recognition of various attack types.

[0080] In the process of obtaining type recognition results by recognizing image sample features and texture images of different dimensions through an initial classifier, if the image sample features and texture images can be better interacted at the information level, a more accurate type recognition result can be obtained. In one possible implementation, in S204, the server determines the first type recognition result based on the first image sample features and the first texture image through the initial classifier, specifically including: S31. The server maps the features of the first image sample to the feature space of the first texture image to obtain the mapped sample features; S32. The server determines a sample attention map based on the features of the mapped sample and the feature distribution of the first texture image for the same image region. The sample attention map is used to identify the attention weight of the image region in the image sample for the object type under the guidance of the features of the mapped sample and the feature distribution of the first texture image. S33. The server generates corresponding attention sample features and attention texture images based on the mapped sample features and the first texture image through the sample attention map; S34. The server determines the first type of recognition result based on the attention sample features and attention texture image through the initial classifier.

[0081] Since the first image sample features refer to a series of information reflected in the feature dimension of the sample image, and the first texture image refers to the visual information in the image reflected in the texture dimension of the sample image, and because they are not in the same feature space, in order to enable information-level interaction between the first image sample features and the first texture image in the same feature space, the server can map the first image sample features to the feature space of the first texture image. This yields mapped sample features located in the feature space of the first texture image. The mapped sample features and the first image sample features have the same image information; both can reflect the content of the image sample in the feature dimension, such as... Figure 4 As shown, Figure 4In this diagram, F represents the first image sample feature, and T represents the first texture image. F can be projected onto a space with the same channel dimension as T using a 1×1 convolutional neural network h1, i.e., projecting F onto the feature space of T to obtain the mapped sample features. .

[0082] The mapped sample features and the first texture image, which are located in the same feature space, have corresponding feature distributions for the image regions of the sample image. Since the mapped sample features and the first texture image reflect the content of the image regions from different dimensions, the feature distributions of the mapped sample features for the same image region and the feature distributions of the first texture image for the same image region are usually different. For example, when the feature distribution of the mapped sample features for a certain image region has more feature information, the feature distribution of the first texture image for that image region may have more feature information or less feature information.

[0083] To address this, the server can determine a sample attention map that comprehensively reflects the feature distribution of the image sample based on the mapped sample features and the feature distribution of the first texture image for the same image region. This sample attention map identifies the attention weight of the image region in the image sample for the object type under the guidance of the mapped sample features and the feature distribution of the first texture image. This attention weight helps the initial classifier to better classify and identify the image based on the mapped sample features and the first texture image. For example, when the mapped sample features have a lot of feature information for a certain image region, and the first texture image also has a lot of feature information for that image region, the attention weight of that image region reflected in the sample attention map is higher. The higher this attention weight, the more attention will be allocated when the initial classifier identifies the object type in subsequent steps, thus making the corresponding category identification result more accurate. In other words, by allowing the mapped sample features and the first texture image to interact at the information level, an attention map that can guide the initial classifier's identification is obtained. Figure 4 As shown, it is possible to map sample features The attention map A is obtained by performing matrix multiplication with the first texture image T.

[0084] After obtaining the sample attention map, the server can generate attention sample features corresponding to the mapped sample features and an attention texture image corresponding to the first texture image. The resulting attention sample features are based on the mapped sample features, supplemented with the feature information of the first texture image through the sample attention map, so that the attention sample features carry the feature information of the first texture image. The resulting attention texture image is based on the first texture image, supplemented with the feature information of the mapped sample features through the sample attention map, so that the attention texture image carries the feature information of the first sample features. Thus, the mapped sample features and the first texture features achieve feature complementarity, such as... Figure 4 As shown, attention map A can be applied to the mapping sample features respectively. The first texture image T is used as the basis for obtaining the attention texture image through another 1×1 convolutional neural network h2. and attention sample features .

[0085] After obtaining the attention sample features and attention texture image, the initial classifier can combine the attention sample features and attention texture image to obtain the corresponding first-type recognition result, such as... Figure 4 As shown, attention texture images can be merged through cascading operations. and attention sample features As input to the initial classifier, the attention sample features not only contain the features of the first image sample but also carry information from the first texture image, and the attention texture image not only contains information from the first texture image but also carries the features of the first image sample. This allows the initial classifier to classify the image samples more comprehensively, resulting in a more reliable first-type recognition result.

[0086] It should be noted that, in this embodiment, for cross-scene image samples, the server can also map the features of the second image sample to the feature space of the second texture image to obtain mapped cross-scene sample features; then, based on the mapped cross-scene sample features and the feature distribution of the second texture image for the same image region, a cross-scene sample attention map is determined; then, based on the mapped cross-scene sample features and the second texture image, corresponding attention cross-scene sample features and attention cross-scene texture images are generated through the cross-scene sample attention map; finally, based on the attention cross-scene sample features and attention cross-scene texture images, the second type of recognition result is determined through the initial classifier.

[0087] For image samples, by combining the attention mechanism of feature dimension and image texture dimension, the features of the first image sample and the first texture image can achieve information-level interaction, resulting in an attention texture image carrying the features of the first image sample and attention sample features carrying the features of the first texture image. Using the attention texture image and attention sample features as input to the initial classifier can effectively improve the data quality of the input to the initial classifier, thereby improving the accuracy of the corresponding category recognition results.

[0088] When fusing information from different dimensions based on the attention mechanism, in order to obtain sample attention maps that reflect more detailed information from a finer-grained perspective, one possible implementation of this method includes: S41. The server divides the image sample into N sub-image regions based on the image region, where N>1; In S11, the server maps the features of the first image sample to the feature space of the first texture image to obtain the mapped sample features, specifically including: S42. The server determines N sub-features corresponding to the features of the first image sample based on the N sub-image regions, and maps the N sub-features to the feature space of the first texture image to obtain N sub-mapping features that make up the mapped sample features. For the i-th sub-image region among N sub-image regions, the server in S32 determines the sample attention map based on the mapped sample features and the feature distribution of the first texture image for the same image region, specifically including: S43. The server obtains the i-th sub-mapping feature corresponding to the i-th sub-image region in the mapping sample features, and the i-th texture mesh corresponding to the i-th sub-image region in the first texture image; S44. The server determines the sub-attention map for the i-th sub-image region in the sample attention map based on the i-th sub-mapping feature and the feature distribution of the i-th texture mesh.

[0089] Since the feature information contained in different image regions in an image sample is not uniform, and the first image sample feature reflects the image features of the entire image region in the image sample, and the first texture image reflects the texture image of the entire image region in the image sample, if the sample attention map corresponding to the entire image region of the image sample is directly determined based on the first image sample and the first texture image corresponding to the entire image region, it may cause the obtained sample attention map to lose some detailed information of the image sample.

[0090] To address this, the server can first divide the image sample into multiple sub-image regions based on the image region. That is, it first partitions the image sample. The finer the partitioning, the more sub-image regions are obtained. This allows the recognition model to better notice the detailed information of the image sample in subsequent steps. However, the computational load of the model will also increase accordingly. Therefore, the number of sub-image regions can be set by relevant technical personnel as needed. Furthermore, during the partitioning process, the image region can be divided uniformly or non-uniformly; there is no limitation here. When uniformly partitioning the image region, it can be divided into a grid of size P×P, where the value of P is set as needed. For example, P can be 2, 4, 8, etc. Figure 5 As shown, the value of P can be 3, dividing the image region into a 3×3 grid.

[0091] After dividing the image region into multiple sub-image regions, the server can map the features of the first image sample to the feature space of the first texture image based on these multiple sub-image regions. Specifically, firstly, based on the multiple sub-image regions, multiple sub-features corresponding to each sub-image region are determined from the features of the first image sample. Sub-features refer to the image features corresponding to each sub-image region; that is, the features of the first image sample are first divided. Then, the multiple sub-features corresponding to each sub-image region are mapped to the feature space of the first texture image, resulting in multiple sub-mapping features corresponding to each sub-image region. Sub-mapping features refer to the mapped image features corresponding to each sub-image region. This lays the foundation for subsequent processing of the first texture image and the mapped sample features based on the multiple sub-image regions in the feature space of the first texture image. Figure 5 As shown, the first image sample feature F can be divided into multiple sub-features based on a 3×3 grid. The i-th sub-feature corresponding to the i-th sub-image region can be... This means that, then, a 1×1 convolutional neural network h1 can be used to... Projected onto a space with the same channel dimension as the first texture image T, that is... Projecting onto the feature space of T, we obtain the i-th sub-mapping feature corresponding to the i-th sub-image region. .

[0092] After obtaining the sub-mapping features corresponding to the multiple sub-image regions, the server can determine the sample attention map based on these multiple sub-image regions. Taking the i-th sub-image region as an example, we can first obtain the i-th sub-mapping feature corresponding to the i-th sub-image region, and the i-th texture mesh in the first texture image corresponding to the i-th sub-image region. Here, the i-th sub-mapping feature refers to the mapping image feature corresponding to the i-th sub-image region, and the i-th texture mesh refers to the texture image corresponding to the i-th sub-image region. Then, based on the feature distribution of the i-th sub-mapping feature and the i-th texture mesh, we determine the i-th sub-attention map that can reflect the feature distribution of the i-th sub-image region from two dimensions, such as... Figure 5 As shown, this can be achieved by modifying the i-th sub-mapping feature. and the i-th texture mesh Matrix multiplication is performed to obtain the sub-attention map. .

[0093] After dividing the image into sub-image regions, the server can determine the corresponding sub-attention maps based on the divided sub-image regions. The sub-attention map refers to the attention map corresponding to the sub-image region. Specifically, it refers to the attention weight of the sub-image region for the object type to be identified under the guidance of the feature distribution of the sub-mapping features and the sub-texture mesh. In this embodiment, instead of directly obtaining the attention weight of the entire image region of the image sample, the attention weight of each sub-image region is obtained by partitioning the image. This allows the features of the first image sample and the first texture image to achieve more detailed information-level interaction according to the sub-image regions. This enables the initial classifier to notice which features of the sub-image regions in the image sample are more relevant to object type recognition. In other words, it allows the initial classifier to better notice the detailed information related to object type recognition in the image sample, thereby improving the recognition accuracy.

[0094] Furthermore, after obtaining the sub-attention map, the server can first generate sub-attention sample features corresponding to the sub-mapping features, and sub-attention texture meshes corresponding to the sub-texture meshes, such as... Figure 5 As shown, the i-th sub-attention graph can be... Element-wise product is applied to the i-th mapping feature. and the i-th texture mesh The features of the i-th sub-attention sample are obtained through residual connections. and the i-th sub-attention texture mesh For example, it can be obtained using the following formula. and :

[0095]

[0096] in, This represents the i-th sub-attention texture mesh. This indicates that α is the average value over a certain dimension. Represents a sub-attention map. Indicates the Kronecker product. Represents the i-th texture mesh. Let i represent the features of the i-th sub-attention sample. This represents a 1×1 convolutional neural network used to project the input onto a matrix. Spaces with the same number of channel dimensions Represents the feature of the i-th sub-mapping. Let i represent the i-th sub-feature.

[0097] Then, attention sample features are reassembled based on the sub-attention sample features, and attention texture images are reassembled based on the sub-attention texture mesh, such as... Figure 5 As shown, the features of the sub-attention samples And the attention texture mesh Reassembling to obtain attention sample features and attention texture image And merged through cascading operations, and As input to the initial classifier.

[0098] It should be noted that, in this embodiment, for cross-scene image samples, the server may first divide the image region of the cross-scene image sample into multiple sub-cross-scene image regions; then, based on the multiple sub-cross-scene image regions, the features of the second image sample are mapped to the feature space of the second texture image to obtain multiple sub-cross-scene mapping features; then, based on the multiple sub-cross-scene mapping features and the cross-scene texture mesh corresponding to the multiple sub-image regions, the corresponding sub-cross-scene attention maps are determined for the multiple sub-image regions respectively.

[0099] For image samples, by first dividing the image into sub-image regions and then determining the attention weights, the features of the first image sample and the first texture image can achieve more detailed information interaction according to the sub-image regions. From a more granular perspective, the initial classifier can pay more attention to which features of the sub-image regions in the image sample are more related to object type recognition. In other words, the initial classifier can better notice the detailed information related to object type recognition in the image sample, further improving the recognition accuracy of the model.

[0100] Therefore, for object type recognition in images, the initial recognition model to be trained includes an initial encoder, an initial decoder, and an initial classifier. The initial encoder determines the first image sample features of the image sample. To improve the model's ability to recognize the same object in different object type recognition environments, a second image sample feature is generated based on the first image sample feature to identify different recognition environments. The initial decoder can obtain corresponding texture images for both image sample features. Based on the image sample features and their corresponding texture images, the initial classifier determines the recognition results for the corresponding image sample and the cross-scene image sample, respectively. The initial recognition model is then trained based on the recognition labels and the recognition loss function generated from the recognition results to obtain the final recognition model. Because of the cross-scene transformation of image sample features, the initial recognition model learns how to recognize the same object in different object type recognition environments during training. Furthermore, this cross-scene transformation avoids encoding additional samples and does not substantially affect the model's training efficiency. In addition to relying on image sample features, texture images were further introduced when performing object type recognition. This allowed for a richer representation of the image from both the feature dimension and the texture dimension, enabling the initial recognition model to learn more accurate recognition capabilities during training and cope with more diverse recognition attack methods. The recognition model trained in this way achieved good generalization ability in cross-scene recognition and various attack types when performing object recognition, effectively improving the model's recognition accuracy.

[0101] In this application, the type recognition result of the initial recognition model is measured by a recognition loss function. That is, the difference between the model prediction value (type recognition result) and the actual value (recognition label) of the initial recognition model is evaluated by the recognition loss function. The initial recognition model is trained by the recognition loss function to obtain a recognition model with a certain recognition accuracy.

[0102] To further improve the recognition accuracy of the recognition model, other types of loss functions can be introduced to train the initial recognition model together with the recognition loss function. In one possible implementation, image samples include positive and negative samples, where objects in positive samples are real objects and objects in negative samples are fake objects. For the texture image generated in S203, the method also includes: The server generates a texture loss function based on the difference between the first texture image of the positive sample and the texture label of the positive sample; In S206, the server trains an initial recognition model using a recognition loss function to obtain the recognition model, which specifically includes: The server trains an initial recognition model using an identification loss function and a texture loss function to obtain the recognition model.

[0103] Image samples refer to images that can be used as training samples for a model. For the initial recognition model, positive samples are image samples containing real objects, while negative samples are image samples containing fake objects. For example, when the recognition model identifies liveness, positive samples contain real live objects, while negative samples can contain fake faces such as video screenshots or photos of faces that are not real live objects. If the initial recognition model is trained using only positive samples, the false detection probability and false recognition rate of the resulting model will be relatively high. Therefore, to reduce the false detection probability and false recognition rate, the image samples should include both positive and negative samples.

[0104] During the initial recognition model training, both positive and negative samples obtain corresponding recognition loss functions through the initial encoder, initial classifier, and initial recognizer in the initial recognition model, thereby obtaining a recognition model with a certain recognition capability. That is, in S203, both positive and negative samples obtain corresponding texture images through the initial decoder. At this time, in this embodiment, in order to enable the recognition model to better distinguish between real and fake object types, the server only measures the generation accuracy of the first texture image corresponding to the positive sample. In order to avoid the information of negative samples interfering with the generation of the texture loss function involving only positive samples, instance normalization (IN) can be used to process individual image samples, instead of batch normalization (BN) commonly used in convolutional neural networks to process the same batch of image samples. Specifically, the server can generate a texture loss function based on the difference between the first texture image of the positive sample and the texture label of the positive sample. Here, the texture label refers to the texture image expected to be generated for the corresponding positive sample, that is, the texture image that the server expects the initial recognition model to generate that can fully reflect the information of the positive sample in the texture dimension. For example, for the batch image samples of the liveness detection model. ,in, This represents the number of image samples in a batch. Let the height and width dimensions of the image sample be denoted as . For a liveness detection model, positive samples are those containing real live objects, while negative samples are those containing false non-live objects. The texture loss function of the liveness detection model can be obtained using the following formula:

[0105] in, Represents the texture loss function. Let E represent the positive samples in the batch of image samples, and D represent the processing corresponding to the initial encoder and the initial decoder, respectively. This represents the first texture image corresponding to the positive sample. Texture labels representing positive samples. Let α be the norm of α.

[0106] This formula evaluates the difference between the first texture image obtained from the initial encoder and initial decoder for positive samples and the texture label of the positive samples, thus measuring the generation accuracy of the first texture image for positive samples.

[0107] It should be noted that although this application also generates corresponding second texture images for cross-scene image samples corresponding to positive samples, in order to ensure the training efficiency of the model, the server directly generates the second image sample features through the features of the first image sample, rather than first performing cross-scene transformation on the image samples to obtain the corresponding cross-scene image samples, and then determining the corresponding second image sample features through the cross-scene image samples. If the generation accuracy of the second texture image is also to be measured, it is necessary to first generate cross-scene image samples based on the features of the second image sample, and then generate corresponding cross-scene texture labels based on the cross-scene image samples, so that the generation accuracy of the second texture image can be evaluated based on the cross-scene texture labels. This will undoubtedly increase the computational load of the model. Furthermore, in this application, the parameters used by the initial decoder to generate the first texture image and the second texture image are the same. Therefore, since the generation accuracy of the first texture image has already been evaluated, it is not necessary to evaluate the generation accuracy of the second texture image as well.

[0108] After obtaining the recognition loss function, the server can train the initial recognition model together with the texture loss function. Since the recognition loss function is obtained by the difference between the first texture image of the positive sample and the texture label of the positive sample, it only involves positive samples and does not involve negative samples. That is, the recognition loss function is asymmetric. Training the initial recognition model with this asymmetric recognition loss function allows the trained recognition model to better distinguish the differences in texture images between positive and negative samples. Since the recognition model will encounter various types of attacks on the images to be processed in actual use, and even new attack types different from the negative samples in the training process, by enabling the recognition model to better distinguish the differences in texture images between positive and negative samples, the recognition model can still distinguish that the new attack type is not the type of object that the recognition model needs to recognize when it encounters a new attack type different from the negative sample, thereby improving the generalization ability of the recognition model when facing various attack types.

[0109] By using the difference between the first texture image of a positive sample and the texture label of the positive sample, a texture loss function is generated to measure the generation accuracy of the texture image of the positive sample. This asymmetric texture loss function and the recognition loss function are used together to train the initial recognition model, so that the trained recognition model can better distinguish the differences between the texture images of positive and negative samples. This is beneficial to improving the generalization ability of the recognition model in the face of various attack types and effectively improving the recognition accuracy of the recognition model.

[0110] In one possible implementation, the texture label of a positive sample can be obtained as follows: The positive samples are subjected to image texture transformation to obtain texture labels.

[0111] Texture labels refer to the texture images expected to be generated for positive samples. That is, the server expects the initial recognition model to generate texture images that can fully reflect the texture information of the positive samples. In practical applications, image texture conversion operations can be performed directly on the positive samples themselves to obtain the corresponding texture labels. Image texture conversion operations refer to performing texture conversion on the image itself to obtain texture images that can fully reflect the texture information of the image. By directly performing image texture conversion on the positive samples, the information loss caused by other operations is minimized, ensuring the reliability of the obtained texture labels. For example, the texture image can be directly processed into grayscale to obtain the corresponding grayscale image, which can be used as a texture label.

[0112] By performing image texture transformation on the positive samples themselves, we can obtain texture labels that fully reflect the information of the positive samples in the texture dimension, so that these texture labels can be used as the basis for calculating the texture loss function.

[0113] To further improve the recognition accuracy of the recognition model, in addition to the texture loss function, other types of loss functions can be introduced to train the initial recognition model together with the recognition loss function. In one possible implementation, image samples include positive samples and negative samples, where objects in positive samples are real objects and objects in negative samples are fake objects. The method also includes: The server obtains the first undetermined features of positive samples and the second undetermined features of negative samples through the initial encoder; The server generates a distance loss function based on the first difference between the first undetermined feature and the anchor feature, and the second difference between the second undetermined feature and the anchor feature. The anchor feature is determined based on positive samples. In S206, the server trains an initial recognition model using a recognition loss function to obtain the recognition model, which specifically includes: The server trains an initial recognition model using an identification loss function and a distance loss function to obtain the recognition model. The initial recognition model is trained using a distance loss function, based on the optimization objective of minimizing the first difference and maximizing the second difference.

[0114] To reduce the false detection probability and false recognition rate of the recognition model, the image samples include positive samples and negative samples. Positive samples are image samples containing real objects, while negative samples are image samples containing fake objects. Both positive and negative samples are used in the initial training of the recognition model.

[0115] Undetermined features refer to the feature information obtained by the server from image samples through the initial encoder. The first undetermined feature is the undetermined feature obtained from positive samples by the initial encoder, and the second undetermined feature is the undetermined feature obtained from negative samples by the initial encoder. Since the objects included in the positive samples are all real object types, the distribution difference of the first undetermined features corresponding to the positive samples will be small, that is, the distribution of the first undetermined features in the feature space will be relatively compact. However, the collection methods of negative samples are usually diverse. For example, when the recognition model recognizes liveness, negative samples include video screenshots, photos, and various other attack methods. That is, the second undetermined features corresponding to the negative samples will be difficult to aggregate together. Furthermore, although the object types included in the positive and negative samples are different... The first undetermined feature corresponding to a positive sample and the second undetermined feature corresponding to a negative sample may have different parts. However, since positive and negative samples may have similar parts in terms of object type recognition environment, for example, when the same acquisition device is used to acquire positive and negative samples indoors, the positive and negative samples will at least have similar features in terms of indoor environment and acquisition device parameters. That is, the boundary between the first and second undetermined features in the feature space is not clear. At this time, if the feature distribution distance of positive samples can be further aggregated and the feature distribution distance between positive and negative samples can be widened, thereby generating a more discriminative and distinguishable feature space, it will help improve the generalization ability of the recognition model when facing various attack types.

[0116] To address this, the server can generate an asymmetric distance loss function based on the first difference between the first undetermined feature and the anchor feature, and the second difference between the second undetermined feature and the anchor feature. The anchor feature refers to the feature information obtained from the sample used as the anchor. Since this embodiment aims to aggregate the feature distribution of positive samples and differentiate the feature distributions of positive and negative samples, the anchor point is determined from the positive samples. This anchor point can be randomly selected from the positive samples. The first difference refers to the difference between the first undetermined feature and the anchor feature corresponding to the positive sample, specifically the distance between the first undetermined feature and the anchor feature in the feature space. The second difference refers to the difference between the second undetermined feature and the anchor feature corresponding to the negative sample, specifically the distance between the second undetermined feature and the anchor feature in the feature space. In this embodiment, suitable distance metrics such as Euclidean distance or cosine distance can be used. The server can generate a distance loss function based on the first and second differences to comprehensively evaluate these differences. For example, when the undetermined feature... When the feature vector is the output of the initial encoder after global average pooling, the distance loss function can be determined by the following formula:

[0117]

[0118]

[0119] in, Let δ denote the distance loss function, and let δ denote the softplus function. These represent the anchor point feature, the first undetermined feature, and the second undetermined feature, respectively. As a distance metric, Indicates the first difference, Indicates the second difference. represent the weighting parameters of the first difference and the second difference, respectively, and Softmax represents the normalization function.

[0120] The above formula uses the distance in the feature space between the anchor feature and the first undetermined feature as the first difference, and the distance in the feature space between the anchor feature and the second undetermined feature as the second difference. Since the weighting parameter normalizes the first or second difference, adding corresponding weighting parameters before the first and second differences allows for dynamic allocation of more importance to difficult samples. The difference between the weighted first difference for all positive samples and the weighted second difference for all negative samples is then processed using the softplus function to obtain the distance loss function. When the distance loss function output by the softplus function is close to zero, the difference between the weighted first difference for all positive samples and the weighted second difference for all negative samples will be a large negative number. This means the weighted first difference will be close to zero and the weighted second difference will be a large positive number. This indicates that the difference between the first undetermined feature and the anchor feature is small, while the difference between the second undetermined feature and the anchor feature is large; that is, the distance between positive samples and the anchor is small, and the distance between negative samples and the anchor is large.

[0121] After obtaining the distance loss function, the server can train the initial recognition model by jointly identifying the loss function and the distance loss function. During the training process, the optimization objective is to minimize the first difference and maximize the second difference. That is, by reducing the first difference between the first undetermined feature and the anchor feature, and expanding the second difference between the second undetermined feature and the anchor feature, the feature distribution distance of positive samples is gathered and the feature distribution distance between positive samples and negative samples is widened.

[0122] It should be noted that this embodiment does not require the convergence of feature distribution distances of negative samples. This is because the attack types in negative samples are not entirely the same, and it would be difficult to converge the feature distributions of negative samples with different feature distributions. Moreover, for negative samples that include various types of fake objects, determining a general feature distribution region would also affect the recognition accuracy of the trained recognition model when facing various attack types in actual use. Therefore, by converging the feature distribution distances of positive samples and widening the feature distribution distances of positive and negative samples, a clearer boundary can be generated between the feature distributions of positive and negative samples in the feature space, which can effectively improve the recognition accuracy of the recognition model.

[0123] By using the first difference between the first undetermined feature of a positive sample and the first difference between the anchor feature determined based on the positive sample, and the second difference between the second undetermined feature of a negative sample and the anchor feature, an asymmetric distance loss function can be generated. This distance loss function, together with the recognition loss function, is used to train the initial recognition model. During training, the distance loss function minimizes the first difference and maximizes the second difference to cluster the feature distribution distance of positive samples and widen the feature distribution distance between positive and negative samples. This results in a more discriminative and distinguishable feature space, which is beneficial to improving the generalization ability of the recognition model in the face of various attack types and effectively improving the recognition accuracy of the recognition model.

[0124] In one possible implementation, the initial encoder includes multiple network layers in addition to the input layer. The first undetermined feature and the second undetermined feature are the output features of the same network layer. When the same network layer is the output layer of the initial encoder, the first undetermined feature and the second undetermined feature are the first image sample features of different image samples.

[0125] In this embodiment, the initial encoder may include multiple network layers in addition to the input layer. Based on the positive or negative samples input to the input layer, any layer in the multiple network layers can determine the corresponding output features for the positive and negative samples respectively. Since the scale of the output features between different layers in the multiple network layers is not the same, according to the needs of model training, the output features of any layer for the positive and negative samples can be used as the first undetermined features and the second undetermined features to generate the distance loss function corresponding to that layer. When the output layer in the multiple network layers generates the first image sample features corresponding to the positive and negative samples respectively, the first image sample features of the output layer for the positive and negative samples can also be directly used as the first undetermined features and the second undetermined features to generate the distance loss function corresponding to the output layer. Furthermore, in order to better train and optimize the initial encoder, the distance loss function can be determined based on the output features of each layer in the multiple network layers, thereby training and optimizing the output features of each layer in the initial encoder at different scales.

[0126] For any one of the multiple network layers in the initial encoder, the corresponding distance loss function can be determined. This allows for the accurate optimization of the network layer to be trained and optimized according to the needs of model training. It is even possible to train and optimize each layer in the initial encoder.

[0127] To further improve the recognition accuracy of the recognition model, in addition to the texture loss function and the distance loss function, other types of loss functions can be introduced to train the initial recognition model together with the recognition loss function. In one possible implementation, image samples include positive samples and negative samples, where objects in positive samples are real objects and objects in negative samples are fake objects. The method also includes: The server generates a calibration loss function based on the feature similarity parameters between the first image sample features of the positive sample and the second image sample features of the cross-scene image sample corresponding to the positive sample; In S206, the server trains an initial recognition model using a recognition loss function to obtain the recognition model, which specifically includes: The server trains an initial recognition model by identifying a loss function and a calibration loss function to obtain the recognition model. The initial recognition model is trained by the calibration loss function based on the optimization objective of the first image sample features with fixed positive samples.

[0128] To reduce the false detection probability and false recognition rate of the recognition model, the image samples include positive samples and negative samples. Positive samples are image samples containing real objects, while negative samples are image samples containing fake objects. Both positive and negative samples are used in the initial training of the recognition model.

[0129] In S202, the server generates second image sample features that identify different recognition environments based on the first image sample features of the image samples. By using both the first and second image sample features for training the initial recognition model, the initial recognition model can learn how to recognize the same object in different object type recognition environments during training. Since various object type recognition environments are encountered in practical applications of the recognition model, if the initial recognition model can learn invariant features related to the real object type in the positive samples during training, then in practical applications, for images containing real object types, regardless of how the object type recognition environment changes, the recognition model can identify the objects in the image as real object types based on these invariant features. This will improve the recognition model's generalization ability in various recognition environments.

[0130] To address this, the server can use the first image sample features of positive samples as a benchmark to reduce the difference between the second image sample features of corresponding cross-scene image samples and the first image sample features. This facilitates the initial recognition model learning the invariant feature from the less different first and second image sample features. Specifically, the server can generate a calibration loss function based on the feature similarity parameters between the first and second image sample features. The feature similarity parameters represent the degree of similarity between different image samples in the dimension of image features. The feature similarity parameters can be represented by the distance between image features in the feature space. For example, suitable distance metrics such as Euclidean distance and cosine distance can be used to represent the feature similarity parameters. The generated calibration loss function can reduce the difference between the second and first image sample features during the initial recognition model training process. For example, when the recognition model identifies a live object, the difference between the second and first image sample features corresponding to the positive sample can be reduced. With the corresponding second image sample features The calibration loss function can be determined using the following formula:

[0131] in, This represents the calibration loss function. The first image sample feature represents the positive sample, and the second image sample feature represents the cross-scene image sample corresponding to the positive sample. This indicates that the gradient stopping operation is in progress. It is a distance metric.

[0132] Using the above formula, a gradient stopping operation can be performed on the first image sample features to fix the first image sample features unchanged. That is, the first image sample features are used as a benchmark. When the value of the calibration loss function is close to zero, it indicates that the second image sample features and the fixed first image sample features have a high similarity.

[0133] After obtaining the calibration loss function, the server can train the initial recognition model together with the recognition loss function and the calibration loss function. During the training process, the features of the first image sample are fixed, and the calibration loss function is used to reduce the difference between the features of the second image sample corresponding to the cross-scene image sample of the positive sample and the features of the first image sample of the positive sample. This allows the features of the second image sample generated based on the first image sample to retain as much as possible the invariant features of the first image sample features that are related to the recognition of the real object type, further improving the accuracy of cross-scene sample amplification, so that the initial recognition model can learn the invariant features during the training process.

[0134] It should be noted that, in this embodiment, no calibration loss function is generated for negative samples. This is because the attack types of negative samples are diverse, and the invariant features related to identifying fake object types are different for different attack types. It is difficult for the initial identification model to learn the invariant features related to identifying fake object types during the training process. Furthermore, given the various attack types encountered by the model in practical applications, if the initial identification model can learn the invariant features related to identifying real object types well during the training process, the resulting identification model can accurately identify real object types in practical applications.

[0135] By using the feature similarity parameters of the first image sample features of positive samples and the second image sample features of corresponding cross-scene image samples, a calibration loss function can be generated. This calibration loss function, together with the recognition loss function, is used to train the initial recognition model. During training, the first image sample features are used as a benchmark to reduce the difference between the second and first image sample features. This allows the second image sample features generated based on the first image sample to retain as many invariant features as possible in the first image sample features that are related to the type of real object being identified. This improves the accuracy of cross-scene sample amplification, enabling the initial recognition model to learn these invariant features during training. This, in turn, helps improve the recognition model's generalization ability in various recognition environments and effectively enhances the recognition accuracy of the model.

[0136] To further improve the recognition accuracy of the recognition model, one possible implementation is to train the initial recognition model using a combination of recognition loss function, texture loss function, distance loss function, and calibration loss function, such as... Figure 6As shown, during training, firstly, image samples, including positive and negative samples, are passed through an initial encoder E to obtain first image sample features, where a distance loss function can be applied to each layer of the initial encoder E. Secondly, second image sample features can be generated from the first image sample features, thereby achieving cross-scene transformation of image samples. After obtaining the second image sample features, on the one hand, based on the first and second image sample features, the initial decoder D can generate corresponding first and second texture images, where the first texture image based on positive samples can generate a texture loss function for model training; on the other hand, based on the first image sample... The first and second image sample features are generated through the initial feature embedding module B. Then, an attention mechanism is used to complement the information of the texture image and the embedding features. Here, the details can be enhanced by dividing the image region of the image sample. Next, the difference between the first embedding feature and the second embedding feature of the corresponding cross-scene image sample is reduced by calibrating the loss function, using the first embedding feature of the positive sample as a benchmark. Finally, based on the embedding features and texture image, the corresponding category recognition result is obtained through the initial classifier, and the recognition loss function used for model training is obtained according to the category recognition result and the recognition label.

[0137] During training, the server can combine the recognition loss function, texture loss function, distance loss function, and calibration loss function to obtain the total loss function for model training. For example, this total loss function can be obtained using the following formula:

[0138] in, This represents the recognition loss function. Represents the texture loss function. Represents the distance loss function. This represents the calibration loss function. These are the weights of the texture loss function, distance loss function, and calibration loss function, respectively.

[0139] Using the above formula, the total loss function can be obtained by recognizing the loss function, texture loss function, distance loss function, and calibration loss function. The initial recognition model can then be trained using this total loss function, resulting in a recognition model that has good generalization ability for cross-scene recognition and recognition of various attack types.

[0140] like Figure 7As shown, the framework of the recognition model with a certain generalization ability generates a corresponding texture image for the input image, rather than directly recognizing and detecting the image. By reconstructing the texture image, the model can recognize the image in both the feature dimension and the texture image dimension. The resulting recognition model has high generalization ability when facing cross-scene recognition and new attack types. To verify the generalization ability of the recognition model, the server can test the recognition model by using new recognition scenarios and new attack types different from those used during training. Specifically, when the initial recognition model is trained using samples from the MSU database, the model can be tested using samples from the OULU database. The image samples in the MSU database and the OULU database have different recognition scenarios. When the initial recognition model is trained using the attack method of curved photos, the model can be tested using the attack method of video replay. The recognition model obtained in this embodiment has good results under the tests of new recognition scenarios and new attack types, indicating that the recognition model obtained in this embodiment has good generalization ability across scenes and new attack types.

[0141] In the foregoing Figure 1-7 Based on the corresponding embodiments, Figure 8 This is a schematic diagram of a model determination device provided in an embodiment of this application. The model determination device 800 includes a first determination unit 801, a first generation unit 802, an acquisition unit 803, a second determination unit 804, a second generation unit 805, and a training unit 806. The first determining unit 801 is used to determine the first image sample features of the image sample through the initial encoder of the initial recognition model. The image sample has an identification label that identifies the object type of the object in the image sample. The initial recognition model further includes an initial decoder and an initial classifier. The first generation unit 802 is used to generate second image sample features based on the first image sample features. The cross-scene image sample corresponding to the second image sample features has a different object type recognition environment than the image sample. The cross-scene image sample and the image sample have the same recognition label. The acquisition unit 803 is used to obtain, through an initial decoder, a first texture image corresponding to an image sample and a second texture image corresponding to a cross-scene image sample based on the features of the first image sample and the features of the second image sample. The second determining unit 804 is used to determine a first type recognition result by an initial classifier based on the features of the first image sample and the first texture image; and to determine a second category recognition result by an initial classifier based on the features of the second image sample and the second texture image. The second generation unit 805 is used to generate a recognition loss function based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, respectively. Training unit 806 is used to train an initial recognition model through a recognition loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

[0142] In one possible implementation, the second generation unit 805 is further configured to generate a texture loss function based on the difference between the first texture image of the positive sample and the texture label of the positive sample when the image sample includes positive samples and negative samples, the objects in the positive samples are real object types and the objects in the negative samples are fake object types. Training unit 806 is used to train an initial recognition model using the recognition loss function and the texture loss function to obtain the recognition model.

[0143] In one possible implementation, the acquisition unit 803 is also used to perform image texture conversion on the positive sample to obtain texture labels.

[0144] In one possible implementation, the second determining unit 804 is used for: The features of the first image sample are mapped onto the feature space of the first texture image to obtain the mapped sample features; Based on the features of the mapped sample and the feature distribution of the first texture image for the same image region, a sample attention map is determined. The sample attention map is used to identify the attention weight of the image region in the image sample for the object type under the guidance of the features of the mapped sample and the feature distribution of the first texture image. Based on the mapped sample features and the first texture image, corresponding attention sample features and attention texture images are generated through the sample attention map; Based on the attention sample features and attention texture image, the first type of recognition result is determined by the initial classifier.

[0145] In one possible implementation, the model determination device 800 further includes a partitioning unit for partitioning the image sample into N sub-image regions based on the image region, where N>1; The second determining unit 804 is used for: Based on N sub-image regions, N corresponding sub-features are determined from the features of the first image sample, and the N sub-features are mapped to the feature space of the first texture image to obtain N sub-mapping features that make up the mapped sample features; For the i-th sub-image region among N sub-image regions, obtain the i-th sub-mapping feature corresponding to the i-th sub-image region in the mapping sample features, and the i-th texture mesh corresponding to the i-th sub-image region in the first texture image; Based on the feature distribution of the i-th sub-map and the i-th texture mesh, determine the sub-attention map for the i-th sub-image region in the sample attention map.

[0146] In one possible implementation, the second generating unit 805 is further configured to: When an image sample includes positive and negative samples, where the objects in the positive samples are real objects and the objects in the negative samples are fake objects, the first undetermined feature of the positive samples and the second undetermined feature of the negative samples are obtained through the initial encoder. Based on the first difference between the first undetermined feature and the anchor feature, and the second difference between the second undetermined feature and the anchor feature, a distance loss function is generated, where the anchor feature is determined based on positive samples. Training unit 806 is used to train an initial recognition model using an identification loss function and a distance loss function to obtain a recognition model. The initial recognition model is trained using a distance loss function based on the optimization objective of minimizing the first difference and maximizing the second difference.

[0147] In one possible implementation, the initial encoder includes multiple network layers in addition to the input layer. The first undetermined feature and the second undetermined feature are the output features of the same network layer. When the same network layer is the output layer of the initial encoder, the first undetermined feature and the second undetermined feature are the first image sample features of different image samples.

[0148] In one possible implementation, the first generating unit 802 is used for: In a batch of M image samples trained on the initial recognition model, for the j-th image sample, determine the feature similarity parameters between the j-th image sample and the remaining M-1 image samples in the M image samples, respectively, in terms of the features of the first image sample. Image samples with feature similarity parameters lower than the similarity threshold among M-1 image samples are identified as differential samples; Based on the first image sample features of the differential samples and the first image sample features of the j-th image sample, generate the second image sample features for the j-th image sample.

[0149] In one possible implementation, the first generating unit 802 is used for: Based on the first image sample features of the differential samples and the first image sample features of the j-th image sample, generate the initial second image sample features; Using the first image sample feature of the j-th image sample as a fusion constraint, the first image sample feature of the j-th image sample and the initial second image sample feature are fused to generate the second image sample feature of the j-th image sample.

[0150] In one possible implementation, the second generation unit 805 is further configured to generate a calibration loss function based on the feature similarity parameters between the first image sample features of the positive sample and the second image sample features of the cross-scene image sample corresponding to the positive sample when the image sample includes positive samples and negative samples, the objects in the positive samples are real object types and the objects in the negative samples are fake object types. The training unit 806 is used to train an initial recognition model through a recognition loss function and a calibration loss function to obtain a recognition model. The initial recognition model is trained through a calibration loss function based on the optimization objective of the first image sample features with fixed positive samples.

[0151] In one possible implementation, the second determining unit 804 is used for: When the initial recognition model also includes an initial feature embedding module, the corresponding first embedding feature is determined by the initial feature embedding module based on the features of the first image sample. Based on the first embedded features and the first texture image, the first type recognition result is determined by the initial classifier.

[0152] Therefore, for object type recognition in images, the initial recognition model to be trained includes an initial encoder, an initial decoder, and an initial classifier. The initial encoder determines the first image sample features of the image sample. To improve the model's ability to recognize the same object in different object type recognition environments, a second image sample feature is generated based on the first image sample feature to identify different recognition environments. The initial decoder can obtain corresponding texture images for both image sample features. Based on the image sample features and their corresponding texture images, the initial classifier determines the recognition results for the corresponding image sample and the cross-scene image sample, respectively. The initial recognition model is then trained based on the recognition labels and the recognition loss function generated from the recognition results to obtain the final recognition model. Because of the cross-scene transformation of image sample features, the initial recognition model learns how to recognize the same object in different object type recognition environments during training. Furthermore, this cross-scene transformation avoids encoding additional samples and does not substantially affect the model's training efficiency. In addition to relying on image sample features, texture images were further introduced when performing object type recognition. This allowed for a richer representation of the image from both the feature dimension and the texture dimension, enabling the initial recognition model to learn more accurate recognition capabilities during training and cope with more diverse recognition attack methods. The recognition model trained in this way achieved good generalization ability in cross-scene recognition and various attack types when performing object recognition, effectively improving the model's recognition accuracy.

[0153] This application also provides a computer device, which is the computer device described above, and may include a terminal device or a server. The model determination device may be configured in the computer device. The computer device will now be described in conjunction with the accompanying drawings.

[0154] If the computer device is a terminal device, please refer to Figure 9 As shown, this application provides a terminal device, taking a mobile phone as an example: Figure 9 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 9 The mobile phone includes components such as a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a Wi-Fi module 1470, a processor 1480, and a power supply 1490. Those skilled in the art will understand that... Figure 9 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0155] The following is combined Figure 9 A detailed introduction to each component of a mobile phone: The RF circuit 1410 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1480; in addition, it transmits uplink data to the base station.

[0156] The memory 1420 can be used to store software programs and modules. The processor 1480 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1420. The memory 1420 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0157] The input unit 1430 can be used to receive input numeric or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432.

[0158] The display unit 1440 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1440 may include a display panel 1441.

[0159] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0160] Audio circuitry 1460, speaker 1461, and microphone 1462 provide an audio interface between the user and the mobile phone.

[0161] WiFi is a short-range wireless transmission technology. Through the WiFi module 1470, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access.

[0162] The processor 1480 is the control center of the mobile phone. It connects various parts of the mobile phone through various interfaces and lines. It performs various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1420 and calling data stored in the memory 1420.

[0163] The phone also includes a power supply 1490 (such as a battery) that powers the various components.

[0164] In this embodiment, the processor 1480 included in the terminal device also has the following functions: The initial encoder of the initial recognition model determines the first image sample features of the image sample. The image sample has an object type identification label that identifies the object in the image sample. The initial recognition model also includes an initial decoder and an initial classifier. The second image sample features are generated based on the features of the first image sample. The cross-scene image sample corresponding to the second image sample features has a different object type recognition environment than the image sample. The cross-scene image sample and the image sample have the same recognition label. Based on the features of the first image sample and the features of the second image sample, the first texture image corresponding to the image sample and the second texture image corresponding to the cross-scene image sample are obtained by the initial decoder; Based on the features of the first image sample and the first texture image, the first type of recognition result is determined by an initial classifier; based on the features of the second image sample and the second texture image, the second category of recognition result is determined by an initial classifier. Based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, a recognition loss function is generated. An initial recognition model is trained by recognizing a loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

[0165] If the computer device is a server, this application embodiment also provides a server; please refer to [link to relevant documentation]. Figure 10 As shown, Figure 10 This is a structural diagram of a server 1500 provided in an embodiment of this application. The server 1500 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1522 (e.g., one or more processors) and memory 1532, and one or more storage media 1530 (e.g., one or more mass storage devices) for storing application programs 1542 or data 1544. The memory 1532 and storage media 1530 can be temporary or persistent storage. The program stored in the storage media 1530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1522 may be configured to communicate with the storage media 1530 and execute the series of instruction operations in the storage media 1530 on the server 1500.

[0166] Server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input / output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0167] The steps performed by the server in the above embodiments can be based on Figure 10 The server structure shown.

[0168] In addition, this application embodiment also provides a storage medium for storing a computer program for executing the method provided in the above embodiment.

[0169] This application also provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to perform the method provided in the above embodiments.

[0170] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk or optical disk, and other media that can store computer programs.

[0171] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0172] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Moreover, based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model determination method, characterized in that, The method includes: The initial encoder of the initial recognition model determines the first image sample features of the image sample, the image sample having an object type identification label that identifies the object in the image sample; the initial recognition model also includes an initial decoder and an initial classifier. A second image sample feature is generated based on the first image sample feature. The cross-scene image sample corresponding to the second image sample feature has a different object type recognition environment than the image sample. The cross-scene image sample has the same recognition label as the image sample. Based on the features of the first image sample and the features of the second image sample, the first texture image corresponding to the image sample and the second texture image corresponding to the cross-scene image sample are obtained by the initial decoder; Based on the features of the first image sample and the first texture image, a first type of recognition result is determined by the initial classifier; based on the features of the second image sample and the second texture image, a second type of recognition result is determined by the initial classifier. Based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, a recognition loss function is generated; The initial recognition model is trained by the recognition loss function to obtain a recognition model, which is used to identify the object type of objects in the image to be processed.

2. The method according to claim 1, characterized in that, The image samples include positive samples and negative samples, wherein the objects in the positive samples are real objects and the objects in the negative samples are fake objects; the method further includes: A texture loss function is generated based on the difference between the first texture image of the positive sample and the texture label of the positive sample; The step of training the initial recognition model using the recognition loss function to obtain the recognition model includes: The initial recognition model is trained using the recognition loss function and the texture loss function to obtain the recognition model.

3. The method according to claim 2, characterized in that, The method further includes: The positive sample is subjected to image texture conversion to obtain the texture label.

4. The method according to claim 1, characterized in that, The step of determining the first type recognition result based on the features of the first image sample and the first texture image using the initial classifier includes: The first image sample features are mapped onto the feature space of the first texture image to obtain the mapped sample features; Based on the mapped sample features and the feature distribution of the first texture image for the same image region, a sample attention map is determined. The sample attention map is used to identify the attention weight of the image region in the image sample for the object type under the guidance of the mapped sample features and the feature distribution of the first texture image. Based on the mapped sample features and the first texture image, corresponding attention sample features and attention texture images are generated through the sample attention map; Based on the attention sample features and the attention texture image, the first type of recognition result is determined by the initial classifier.

5. The method according to claim 4, characterized in that, The method further includes: The image sample is divided into N sub-image regions based on the image region, where N>1; The step of mapping the features of the first image sample onto the feature space of the first texture image to obtain the mapped sample features includes: Based on the N sub-image regions, N corresponding sub-features are determined from the features of the first image sample, and the N sub-features are mapped to the feature space of the first texture image to obtain N sub-mapping features that make up the mapped sample features; For the i-th sub-image region among N sub-image regions, determining the sample attention map based on the mapped sample features and the feature distribution of the first texture image for the same image region includes: Obtain the i-th sub-mapping feature corresponding to the i-th sub-image region in the mapped sample features, and the i-th texture mesh corresponding to the i-th sub-image region in the first texture image; Based on the i-th sub-mapping feature and the feature distribution of the i-th texture mesh, a sub-attention map for the i-th sub-image region is determined in the sample attention map.

6. The method according to claim 1, characterized in that, The image samples include positive samples and negative samples, wherein the objects in the positive samples are real objects and the objects in the negative samples are fake objects; the method further includes: The first undetermined feature of the positive sample and the second undetermined feature of the negative sample are obtained through the initial encoder. Based on the first difference between the first undetermined feature and the anchor feature, and the second difference between the second undetermined feature and the anchor feature, a distance loss function is generated, wherein the anchor feature is determined based on the positive sample; The step of training the initial recognition model using the recognition loss function to obtain the recognition model includes: The initial recognition model is trained using the recognition loss function and the distance loss function to obtain the recognition model, wherein the initial recognition model is trained using the distance loss function based on the optimization objective of minimizing the first difference and maximizing the second difference.

7. The method according to claim 6, characterized in that, The initial encoder includes multiple network layers in addition to the input layer. The first undetermined feature and the second undetermined feature are output features of the same network layer. When the same network layer is the output layer of the initial encoder, the first undetermined feature and the second undetermined feature are first image sample features of different image samples.

8. The method according to claim 1, characterized in that, In a batch of M image samples trained on the initial recognition model, for the j-th image sample, generating second image sample features based on the features of the first image sample includes: Determine the feature similarity parameters between the j-th image sample and the remaining M-1 image samples in the M image samples, respectively, in the first image sample features; Image samples whose feature similarity parameters are lower than the similarity threshold among the M-1 image samples are identified as differential samples; Based on the first image sample features of the differential sample and the first image sample features of the j-th image sample, a second image sample feature is generated for the j-th image sample.

9. The method according to claim 8, characterized in that, The step of generating second image sample features for the j-th image sample based on the first image sample features of the differential samples and the first image sample features of the j-th image sample includes: Based on the first image sample features of the differential samples and the first image sample features of the j-th image sample, an initial second image sample feature is generated; Using the first image sample feature of the j-th image sample as a fusion constraint, the first image sample feature of the j-th image sample and the initial second image sample feature are fused to generate the second image sample feature for the j-th image sample.

10. The method according to claim 1, characterized in that, The image samples include positive samples and negative samples, wherein the objects in the positive samples are real objects and the objects in the negative samples are fake objects; the method further includes: Based on the feature similarity parameters between the first image sample features of the positive sample and the second image sample features of the cross-scene image sample corresponding to the positive sample, a calibration loss function is generated; The step of training the initial recognition model using the recognition loss function to obtain the recognition model includes: The initial recognition model is trained by the recognition loss function and the calibration loss function to obtain the recognition model, wherein the initial recognition model is trained by the calibration loss function based on the optimization objective of the first image sample features of the fixed positive samples.

11. The method according to any one of claims 1-9, characterized in that, The initial recognition model further includes an initial feature embedding module, wherein determining the first type of recognition result based on the first image sample features and the first texture image using the initial classifier includes: Based on the features of the first image sample, the corresponding first embedding feature is determined by the initial feature embedding module; Based on the first embedded feature and the first texture image, the first type recognition result is determined by the initial classifier.

12. A model determining device, characterized in that, The device includes a first determining unit, a first generating unit, an acquiring unit, a second determining unit, a second generating unit, and a training unit. The first determining unit is used to determine the first image sample features of the image sample through the initial encoder of the initial recognition model. The image sample has an object type identification label that identifies the object in the image sample. The initial recognition model further includes an initial decoder and an initial classifier. The first generation unit is configured to generate a second image sample feature based on the first image sample feature, wherein the cross-scene image sample corresponding to the second image sample feature has a different object type recognition environment than the image sample, and the cross-scene image sample has the same recognition label as the image sample; The acquisition unit is used to obtain, through the initial decoder, a first texture image corresponding to the image sample and a second texture image corresponding to the cross-scene image sample based on the first image sample features and the second image sample features. The second determining unit is configured to determine a first type of recognition result based on the first image sample features and the first texture image using the initial classifier; and to determine a second type of recognition result based on the second image sample features and the second texture image using the initial classifier. The second generation unit is used to generate a recognition loss function based on the differences between the first type of recognition result and the second type of recognition result and the recognition label, respectively. The training unit is used to train the initial recognition model through the recognition loss function to obtain a recognition model, which is used to identify the object type of the object in the image to be processed.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to perform the method according to any one of claims 1-11 according to the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1-11.

15. A computer program product comprising a computer program, which, when run on a computer device, causes the computer device to perform the method of any one of claims 1-11.

Images