Robustness compensation method and device for pattern recognition model of unconstrained scene and medium

Abstract

The application discloses a mode recognition model robustness compensation method and device in an unconstrained scene, and a medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a picture collected by an image collection device; acquiring a quality score of the picture and multi-scale information of the picture; fusing the multi-scale information of the picture according to the quality score of the picture; and performing robustness compensation on a mode recognition model in the unconstrained scene according to the fused information of the picture. In the method, the quality score determines the fusion degree when the multi-scale information is fused. In addition, because the information of different scales is different, the image information concerned is different, so that the multi-scale information can be fused to better understand the content of the picture, the semantic information and the texture information are fused, and thus the robustness compensation on the mode recognition model in the unconstrained scene can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a mode recognition model robustness compensation method and device for unconstrained scenes and a medium. BACKGROUND

[0002] With the development of the information age, most scenes need to use mode recognition. Mode recognition includes simple mode recognition and unconstrained mode recognition. Compared with simple mode recognition, unconstrained mode recognition has more diverse scenes, and is not limited to high-definition images, but even rotated images or images with large-area occlusion need to be recognized, which has very high requirements for the robustness of the model.

[0003] Traditional deep learning-based mode recognition training is mainly to learn image information of samples, such as colors and lines, so as to distinguish different classes. Such mode recognition generally focuses on the model to enable the model to better extract sample features, or to increase the data of unconstrained samples to enable the model to learn more robust features. However, due to the difficulty in obtaining data, it is difficult to achieve good results by this method. Taking face recognition as an example, for unconstrained face recognition, the quality of the face will be disturbed by various disturbances such as occlusion, blur, and angle. The traditional face recognition does not distinguish these quality information, resulting in that the traditional face recognition extracts features without distinction when encountering such unconstrained scenes, and these low-quality pictures will inevitably lead to incorrect recognition, thereby affecting the robustness of the model to the unconstrained scene. For example, when solving a case, the investigators determine a suspicious person through monitoring, but the image collected by the image collection device may be a side face, an occluded face, or a blurred face of the suspicious person, and the traditional face recognition method cannot correctly recognize these low-quality pictures, thereby affecting the robustness of the model to the unconstrained scene.

[0004] Therefore, how to compensate the robustness of the mode recognition model for the unconstrained scene is a problem to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide a mode recognition model robustness compensation method, device and medium for unconstrained scenes, to improve the robustness of the model to the unconstrained scene.

[0006] To solve the above technical problems, the present application provides a mode recognition model robustness compensation method for unconstrained scenes, comprising:

[0007] obtaining a picture collected by an image collection device;

[0008] obtaining a quality score of the picture and multi-scale information of the picture;

[0009] fusing multi-scale information of the picture according to a quality score of the picture;

[0010] robustly compensating the pattern recognition model for unconstrained scenes according to the fused information of the picture.

[0011] Preferably, the acquiring the quality score of the picture comprises:

[0012] accessing a first fully connected layer in a neural network structure and generating a first vector; wherein the first vector is a vector representing the picture;

[0013] accessing a second fully connected layer after the first fully connected layer and generating corresponding one-dimensional data;

[0014] training the one-dimensional data as the quality score of the picture through a loss function.

[0015] Preferably, the robustly compensating the pattern recognition model for unconstrained scenes according to the fused information of the picture comprises:

[0016] accessing a third fully connected layer with intermediate arbitrary scale information of the neural network structure as input and generating a corresponding second vector as a robust compensation branch; wherein the dimension of the second vector is the same as that of the first vector;

[0017] compensating the first vector through the robust compensation branch.

[0018] Preferably, after the compensating the first vector through the robust compensation branch, the method further comprises:

[0019] acquiring original features of the picture and features of the robust compensation branch;

[0020] weighting the original features of the picture and the features of the robust compensation branch to obtain output features of the pattern recognition model; wherein the weighting coefficient is calculated through the quality score generated by the second fully connected layer.

[0021] Preferably, after the compensating the first vector through the robust compensation branch, the method further comprises:

[0022] introducing a standard normal distribution in the neural network structure; wherein the standard normal distribution is used to introduce a controllable fluctuation for the output features of the pattern recognition model;

[0023] acquiring original features of the picture and features of the robust compensation branch;

[0024] The output features of the pattern recognition model are obtained based on the original features of the image, the features of the robustness compensation branch, and the standard normal distribution.

[0025] Preferably, the neural network structure is a ResNet network structure.

[0026] Preferably, after performing robust compensation on the pattern recognition model of the unconstrained scene based on the multi-scale information of the fused image, the method further includes:

[0027] Output a prompt message to indicate that robustness compensation has been performed on the pattern recognition model.

[0028] To address the aforementioned technical problems, this application also provides a robustness compensation device for pattern recognition models in unconstrained scenarios, comprising:

[0029] The first acquisition module is used to acquire images captured by the image acquisition device;

[0030] The second acquisition module is used to acquire the quality score of the image and the multi-scale information of the image;

[0031] The fusion module is used to fuse multi-scale information of the image based on the image's quality score;

[0032] The compensation module is used to perform robust compensation on the pattern recognition model for the unconstrained scene based on the information of the fused image.

[0033] To address the aforementioned technical problems, this application also provides a robustness compensation device for pattern recognition models in unconstrained scenarios, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is used to implement the steps of the above-described robust compensation method for pattern recognition models in unconstrained scenarios when executing the computer program.

[0036] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described robust compensation method for pattern recognition models in unconstrained scenarios.

[0037] This application provides a robust compensation method for pattern recognition models in unconstrained scenes. The method includes acquiring images captured by an image acquisition device; obtaining the image's quality score and multi-scale information; fusing the multi-scale information based on the image's quality score; and performing robust compensation on the pattern recognition model for unconstrained scenes based on the information from the fused image. In this method, the quality score determines the degree of fusion during multi-scale information fusion. Furthermore, since different scales focus on different image information, fusing multi-scale information allows for a better understanding of the image's content, integrating semantic and textural information, thereby achieving robust compensation for the pattern recognition model in unconstrained scenes.

[0038] In addition, this application also provides a robust compensation device for pattern recognition models in unconstrained scenarios and a computer-readable storage medium, which correspond to the robust compensation method for pattern recognition models in unconstrained scenarios mentioned above and have the same effect. Attached Figure Description

[0039] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a robustness compensation method for a pattern recognition model in an unconstrained scene, provided as an embodiment of this application;

[0041] Figure 2 This is a schematic diagram of the ResNet network structure provided in this embodiment;

[0042] Figure 3 A schematic diagram illustrating the generation of quality scores in the ResNet network structure provided in this application embodiment;

[0043] Figure 4 A flowchart illustrating a robustness compensation method in a ResNet network structure provided in this application embodiment;

[0044] Figure 5 A schematic diagram illustrating the introduction of characteristic fluctuations in the ResNet network structure provided in this application embodiment;

[0045] Figure 6 A structural diagram of a robustness compensation device for a pattern recognition model in an unconstrained scene provided in an embodiment of this application;

[0046] Figure 7 This is a structural diagram of a robustness compensation device for a pattern recognition model in an unconstrained scene, provided in another embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0048] The core of this application is to provide a robustness compensation method, device, and medium for pattern recognition models in unconstrained scenarios, which can be used to improve the robustness of the model in unconstrained scenarios.

[0049] In practice, unconstrained pattern recognition is often required. Compared to simple pattern recognition, unconstrained pattern recognition handles more diverse scenarios, not limited to high-resolution images, but also requiring the recognition of rotated images or images with large areas of occlusion. This places very high demands on the robustness of the model. In statistics and machine learning, the property of maintaining stability and reliability in the face of outliers is called robustness. Model robustness refers to the model's generalization ability, that is, the ability of the model to provide appropriate outputs for data outside of datasets with the same patterns. In reality, there is uncertainty in the data, namely, various uncertainties inherent in the data itself, such as occlusion, noise, and bias.

[0050] Taking facial recognition as an example, it is one of the hottest research areas in artificial intelligence today. It's an algorithm that identifies individuals based on facial image features. In 2D facial recognition, the currently popular deep learning-based methods extract facial features through deep neural networks, then identify individuals by comparing the similarity of features between two facial images. The existence of data uncertainty often affects the model's output, posing a significant challenge to the model's robustness. For example, occluded, blurred, or side-view images collected during crime-solving may affect the accuracy of facial recognition. Therefore, robustness compensation is needed for facial recognition models in unconstrained scenarios to achieve accurate facial recognition.

[0051] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 A flowchart illustrating a robustness compensation method for pattern recognition models in unconstrained scenarios, as provided in this application embodiment, is shown below. Figure 1 As shown, the method includes:

[0052] S10: Acquire images captured by the image acquisition device.

[0053] Image acquisition devices can be cameras, mobile phones, or other similar devices, used to capture images. These images can be high-resolution, blurry, or obscured. High-resolution images allow for more accurate extraction of target features, leading to more accurate target identification. However, blurry or obscured images may prevent the extraction of target features, making target identification impossible. For example, in crime solving, high-resolution, unobstructed images make target identification easier, while blurry or obscured images hinder accurate target identification, thus complicating the investigation.

[0054] S11: Obtain the image quality score and multi-scale information of the image.

[0055] Traditional deep learning-based pattern recognition training primarily aims to learn image information from samples, such as color and lines, to distinguish different classes. However, for some unconstrained pattern recognition tasks, this can lead to numerous false recognitions. Therefore, this embodiment combines image quality scores and multi-scale information to reduce false recognition.

[0056] For face recognition, a simple way to reflect the quality of a face image is by assessing the accuracy of the model's classification and the probability of a correct classification. Simply put, the more certain the model is about an image belonging to a certain class, the higher the image quality. Conversely, the less certain the model is, the lower the image quality. To derive an image quality score, a quality branch can be introduced into the recognition model. On the other hand, different depths of neural network models have different receptive fields. Fusing features from different depths helps the neural network model distinguish features at different scales, thereby improving the model's accuracy and robustness. Deep residual networks (ResNet) are most commonly used in face recognition. This network doesn't have a very obvious fusion module because common face recognition methods scale faces to a uniform size, so the face doesn't undergo much scale variation. However, even after scaling, some detailed information, such as occlusions, can reveal the quality of the face. Therefore, obtaining multi-scale information from images can better represent the image content. In practice, by taking the features of a certain stage of the image as input into the neural network structure, and then connecting it to a fully connected layer, a vector is generated. This vector is used to represent the quality compensation of the image. This vector can learn the data uncertainty of the image, that is, mainly representing the perturbations such as occlusion and blurring of the image, and can also learn facial features from different scales.

[0057] S12: Fuse multi-scale information of images based on their quality scores.

[0058] When fusing multi-scale information of images based on their quality scores, the model can classify images with high quality scores well without much intervention, resulting in low compensation and a smaller degree of fusion. Conversely, the fusion degree is higher for images with low quality scores.

[0059] S13: Perform robustness compensation on the pattern recognition model for unconstrained scenes based on the information from the fused images.

[0060] The above steps integrate multi-scale information from the images, and the information from the integrated images can be used to better extract some data noise information, thereby compensating for it.

[0061] This embodiment provides a robust compensation method for pattern recognition models in unconstrained scenes. The method includes acquiring images captured by an image acquisition device; acquiring the image's quality score and multi-scale information; fusing the multi-scale information based on the image's quality score; and performing robust compensation on the pattern recognition model in the unconstrained scene based on the information from the fused image. In this method, the quality score determines the degree of fusion during multi-scale information fusion. Furthermore, since different scales of information focus on different image information, fusing multi-scale information allows for a better understanding of the image's content, integrating semantic information, texture information, etc., thereby achieving robust compensation for the pattern recognition model in unconstrained scenes.

[0062] In practice, to obtain the image quality score, a preferred implementation method includes obtaining the image quality score by:

[0063] A first fully connected layer is added to the neural network structure to generate a first vector; where the first vector is a vector representing the image;

[0064] The second fully connected layer is connected after the first fully connected layer to generate the corresponding one-dimensional data;

[0065] One-dimensional data is trained using a loss function to obtain the quality score for the image.

[0066] The placement of the first fully connected layer within the neural network structure is not limited, but preferably, it is placed towards the end of the neural network structure. A second connected layer is then placed after the first fully connected layer to generate corresponding data, which can be used to train an image quality score. Figure 2 This is a schematic diagram of the ResNet network structure provided in this embodiment. Figure 2As shown, the input image is pre-processed to a size of 112*112, then undergoes a first convolution to generate a size of 56*56, a second convolution to generate a size of 28*28, a third convolution to generate a size of 14*14, and a fourth convolution to generate a size of 7*7. Finally, a fully connected layer is applied to generate a 1*512 dimensional vector, which is the first vector used to represent the image. A quality branch is introduced into the model to provide a quality score for the image. At this point, a quality extraction head is needed to generate a numerical value representing the quality. Figure 3 This is a schematic diagram illustrating the generation of quality scores in the ResNet network structure provided in an embodiment of this application. For example... Figure 3 As shown, after the first fully connected layer, a simple fully connected layer is introduced to generate a 1*1 number, which generates the quality score of the image. This generated quality score is constrained by the probability that the image is correctly classified. Since Softmax is typically used as the loss function for face recognition training, the probability P of a face being correctly classified can be calculated based on the properties of Softmax. Since the more certain the model is that the image belongs to a certain class, the higher the quality of the image, it can be approximated that the higher P is, the better the quality. Therefore, a simple L1 Loss (loss function) can be used to train the quality branch, as shown in formula (1).

[0067] Loss = abs(PM) (1)

[0068] In the formula, M represents the score of the quality branch output.

[0069] The method for obtaining image quality provided in this embodiment only requires introducing a very small fully connected layer into the original network to represent the quality score, thus enabling the image quality score to be obtained quickly.

[0070] To enable the model to analyze low-quality images such as blurry or occluded images, robustness compensation for the pattern recognition model in unconstrained scenes is achieved by fusing multi-scale information from the images. A preferred implementation method involves robustness compensation for the pattern recognition model in unconstrained scenes based on the information from the fused images, including:

[0071] The intermediate arbitrary scale information of the neural network structure is used as input to the third fully connected layer to generate a corresponding second vector as a robustness compensation branch; wherein, the dimension of the second vector is the same as the dimension of the first vector;

[0072] The first vector is compensated by a robust compensation branch.

[0073] Figure 4A flowchart illustrating a robustness compensation method in a ResNet network structure provided in this application embodiment. Figure 4 As shown, a robust branch is introduced, which takes 28*28 features as input and is followed by a fully connected layer to generate a 1*512 dimensional vector, the second vector, to represent the quality compensation of the image. We hope it can learn the data uncertainty of the image, mainly representing perturbations such as occlusion and blurring, and can also learn facial features at different scales.

[0074] The robust compensation method for pattern recognition models in unconstrained scenes based on the fused images provided in this embodiment achieves robust compensation simply by introducing a fully connected layer, building upon the previous embodiment. Therefore, the method in this embodiment can quickly and conveniently achieve robust compensation for pattern recognition models in unconstrained scenes.

[0075] In the above embodiments, information from different receptive fields is extracted to compensate for the features of face recognition. However, different receptive fields focus on different information, so model fusion is performed in this embodiment. A preferred implementation further includes, after compensating the first vector through a robust compensation branch:

[0076] Obtain the original features of the image and the features of the robust compensation branch;

[0077] The original features of the image and the features of the robustness compensation branch are weighted to obtain the output features of the pattern recognition model; where the weighting coefficients are calculated from the quality score generated by the second fully connected layer.

[0078] The original features of the image and the features of the robustness compensation branch are weighted using formula (2).

[0079] F = F ori +λ*F qua (2)

[0080] In the formula, F represents the features of the image. ori This represents the original characteristics, F qua This represents the characteristics of the robust compensation branch, where λ represents the weighting coefficient of the robust compensation branch, with a value of (1-M), and M is the output of the quality branch. The worse the quality, the smaller the value of M, and the lower the F... qua The larger the proportion, the more compensation will be provided.

[0081] This embodiment provides a weighted approach to obtaining the model's output features by combining the original features of the image with the features of the robustness compensation branch. Since different receptive fields focus on different information, this method is similar to model fusion based on its own features, thereby increasing the points that the model focuses on. Model fusion is performed without significantly changing the number of model parameters.

[0082] Traditional pattern recognition methods output a fixed set of features for a given image. This training method is not conducive to adapting to data uncertainty. Therefore, a preferred implementation method, after compensating the first vector through a robust compensation branch, further includes:

[0083] A standard normal distribution is generated in the neural network structure; the standard normal distribution is used to introduce a controllable fluctuation in the output features of the pattern recognition model.

[0084] Obtain the original features of the image and the features of the robust compensation branch;

[0085] The output features of the model are obtained based on the original features of the image, the features of the robust compensation branch, and the standard normal distribution.

[0086] In this embodiment, a standard normal distribution Q with a mean of 0 and a variance of 1 is generated to introduce characteristic fluctuations. Figure 5 This is a schematic diagram illustrating the introduction of characteristic fluctuations into the ResNet network structure provided in this application embodiment. For example... Figure 5 As shown, a standard normal distribution Q is introduced into the neural network structure. The final feature output formula is formula (3):

[0087] F = F ori +λ*Q*F qua (3)

[0088] As shown in the above formula, uncertainty can be introduced into the final features. Since Q will change to some extent each time it is generated, the features will also change, which increases the robustness of the output features during training. When using the final features after training, in order to ensure the stability of the effect, the Q parameter is removed. That is, the feature formula obtained in actual use is as shown in the above formula (2).

[0089] The embodiment described herein introduces feature fluctuations after compensating the image through a robustness compensation branch, so that a fixed input image is no longer a fixed output, thereby making the model's features more robust.

[0090] In implementation, there are no restrictions on the neural network model used, but preferably, the neural network structure is a ResNet network structure.

[0091] The ResNet network structure is characterized by its ease of optimization and the ability to improve accuracy by significantly increasing its depth. Its internal residual blocks utilize skip connections, mitigating the vanishing gradient problem inherent in deep neural networks. Therefore, this embodiment employs the ResNet network structure for image compensation. It should be noted that the ResNet network structure used in this embodiment is only a preferred neural network structure. In practice, other neural network structures can also be used to achieve robust compensation for pattern recognition models in unconstrained scenes, all within the scope of this application.

[0092] In the above embodiments, a robustness compensation method for pattern recognition models in unconstrained scenarios has been described in detail. This application also provides embodiments corresponding to a robustness compensation device for pattern recognition models in unconstrained scenarios. It should be noted that this application describes the embodiments of the device from two perspectives: one based on functional modules and the other based on hardware.

[0093] Figure 6 A structural diagram of a robustness compensation device for a pattern recognition model in an unconstrained scene, provided as an embodiment of this application. This embodiment, based on functional modules, includes:

[0094] The first acquisition module 10 is used to acquire images captured by the image acquisition device;

[0095] The second acquisition module 11 is used to acquire the quality score of the image and the multi-scale information of the image;

[0096] Fusion module 12 is used to fuse multi-scale information of images based on their quality scores;

[0097] The compensation module 13 is used to perform robust compensation on the pattern recognition model of the unconstrained scene based on the information of the fused image.

[0098] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0099] The robustness compensation device for pattern recognition models in unconstrained scenes provided in this embodiment acquires images from an image acquisition device through a first acquisition module; acquires the quality score and multi-scale information of the images through a second acquisition module; fuses the multi-scale information of the images based on the quality score through a fusion module; and performs robust compensation for the pattern recognition model in unconstrained scenes based on the information of the fused images through a compensation module. In this device, the quality score determines the degree of fusion during multi-scale information fusion. Furthermore, since different scales of information focus on different image information, fusing multi-scale information allows for a better understanding of the image content, integrating semantic information, texture information, etc., thereby achieving robust compensation for the pattern recognition model in unconstrained scenes.

[0100] Figure 7 This is a structural diagram of a robustness compensation device for a pattern recognition model in an unconstrained scene, provided in another embodiment of this application. This embodiment is based on a hardware perspective, such as... Figure 7 As shown, the robustness compensation device for pattern recognition models in unconstrained scenarios includes:

[0101] Memory 20 is used to store computer programs;

[0102] The processor 21 is used to implement the steps of the robustness compensation method for pattern recognition models in unconstrained scenarios as mentioned in the above embodiments when executing a computer program.

[0103] The robustness compensation device for pattern recognition models in unconstrained scenarios provided in this embodiment can include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0104] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0105] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the robustness compensation method for pattern recognition models in unconstrained scenarios disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the aforementioned robustness compensation method for pattern recognition models in unconstrained scenarios.

[0106] In some embodiments, the robustness compensation device for pattern recognition models in unconstrained scenarios may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0107] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the robustness compensation device for pattern recognition models in unconstrained scenarios and may include more or fewer components than shown.

[0108] The robustness compensation device for pattern recognition models in unconstrained scenarios provided in this application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: robustness compensation method for pattern recognition models in unconstrained scenarios, with the same effect as above.

[0109] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0110] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The computer-readable storage medium provided in this application includes the robustness compensation method for pattern recognition models in unconstrained scenarios mentioned above, with the same effect.

[0112] The foregoing provides a detailed description of a robust compensation method, apparatus, and medium for pattern recognition models in unconstrained scenarios, as provided in this application. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0113] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A robustness compensation method for pattern recognition models in unconstrained scenarios, characterized in that, include: Acquire images captured by the image acquisition device; Obtain the quality score and multi-scale information of the image; Multi-scale information of the images is fused based on their quality scores; where images with high quality scores have a lower degree of fusion, and images with low quality scores have a higher degree of fusion. Robust compensation is performed on the pattern recognition model for the unconstrained scene based on the information from the fused image. The process of obtaining the quality score of the image includes: A first fully connected layer is added to the neural network structure to generate a first vector; wherein the first vector is a vector representing the image; A second fully connected layer is connected after the first fully connected layer to generate corresponding one-dimensional data; The one-dimensional data is trained using a loss function to serve as the quality score for the image.

2. The robustness compensation method for pattern recognition models in unconstrained scenes according to claim 1, characterized in that, The robustness compensation of the pattern recognition model for the unconstrained scene based on the information of the fused image includes: The intermediate arbitrary scale information of the neural network structure is used as input to the third fully connected layer to generate a corresponding second vector as a robustness compensation branch; wherein, the dimension of the second vector is the same as the dimension of the first vector; The first vector is compensated by the robust compensation branch.

3. The robustness compensation method for pattern recognition models in unconstrained scenes according to claim 2, characterized in that, After compensating the first vector through the robust compensation branch, the method further includes: Obtain the original features of the image and the features of the robustness compensation branch; The original features of the image and the features of the robustness compensation branch are weighted to obtain the output features of the pattern recognition model; wherein the weighting coefficient is calculated from the quality score generated by the second fully connected layer.

4. The robustness compensation method for pattern recognition models in unconstrained scenes according to claim 2, characterized in that, After compensating the first vector through the robust compensation branch, the method further includes: A standard normal distribution is introduced into the neural network structure; wherein the standard normal distribution is used to introduce a controllable fluctuation into the output features of the pattern recognition model; Obtain the original features of the image and the features of the robustness compensation branch; The output features of the pattern recognition model are obtained based on the original features of the image, the features of the robustness compensation branch, and the standard normal distribution.

5. The robustness compensation method for pattern recognition models in unconstrained scenes according to claim 1, characterized in that, The neural network structure is a ResNet network structure.

6. The robustness compensation method for pattern recognition models in unconstrained scenes according to any one of claims 1 to 5, characterized in that, After performing robust compensation on the pattern recognition model for the unconstrained scene based on the multi-scale information of the fused image, the method further includes: Output a prompt message to indicate that robustness compensation has been performed on the pattern recognition model.

7. A robustness compensation device for pattern recognition models in unconstrained scenes, characterized in that, include: The first acquisition module is used to acquire images captured by the image acquisition device; The second acquisition module is used to acquire the quality score of the image and the multi-scale information of the image; The fusion module is used to fuse multi-scale information of the images based on their quality scores; wherein, for images with high quality scores, the fusion degree is small, and for images with low quality scores, the fusion degree is large. The compensation module is used to perform robust compensation on the pattern recognition model for the unconstrained scene based on the information of the fused image. The process of obtaining the quality score of the image includes: A first fully connected layer is added to the neural network structure to generate a first vector; wherein the first vector is a vector representing the image; A second fully connected layer is connected after the first fully connected layer to generate corresponding one-dimensional data; The one-dimensional data is trained using a loss function to serve as the quality score for the image.

8. A robustness compensation device for pattern recognition models in unconstrained scenes, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the robustness compensation method for pattern recognition models in unconstrained scenarios as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the robustness compensation method for pattern recognition models in unconstrained scenarios as described in any one of claims 1 to 6.

Images