Image processing method, apparatus and device, and non-transitory storage medium

Abstract

The application discloses an image processing method, device and equipment, and a nonvolatile storage medium. The method comprises the following steps: obtaining a sample image, wherein the sample image comprises an original input image and a target input image, and the target input image is obtained by performing data enhancement processing on the original input image; performing feature extraction processing on the sample image by using a neural network model to obtain a feature vector; obtaining a feature mean value corresponding to the feature vector; and obtaining a target feature based on the feature vector and the feature mean value. The application solves the technical problem that, in the prior art, due to the existence of a shielding image and / or an incomplete image, the recognition accuracy of an image processing algorithm is low.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method, apparatus and device, and a non-volatile storage medium. Background Technology

[0002] In related technologies, moving object re-identification technology refers to retrieving images of the same moving object from different devices, given a monitored image of the moving object. Due to differences in different camera devices, installation environments, and the poses of the moving object, moving object re-identification is easily affected by factors such as background, viewing angle, pose, resolution, and occlusion.

[0003] For the occlusion recognition problem, existing methods can be roughly divided into two categories. The first category is occlusion recognition methods based on pose recognition and graph matching; the second category is feature pyramid matching methods. Both of these existing recognition algorithms suffer from the technical problems of the recognition accuracy and detection performance of the algorithm model being drastically reduced due to the influence of occlusion and incompleteness caused by moving objects commonly present in offline scenes, as well as the excessive complexity of using multi-layer feature matching or graph matching, which increases the computational cost.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides an image processing method, apparatus, device, and non-volatile storage medium to at least solve the technical problem in the prior art where the recognition accuracy of image processing algorithms is low due to the presence of occluded and / or incomplete images.

[0006] According to one aspect of the embodiments of this application, an image processing method is provided, comprising: acquiring a sample image, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring a target feature based on the feature vector and the feature mean.

[0007] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, comprising: a first acquisition module, configured to acquire a sample image, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; an extraction module, configured to perform feature extraction processing on the sample image using a neural network model to obtain a feature vector; a second acquisition module, configured to acquire the feature mean corresponding to the feature vector; and a third acquisition module, configured to acquire a target feature based on the feature vector and the feature mean.

[0008] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute any of the above-described image processing methods.

[0009] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, including: a processor; and a memory connected to the processor, configured to provide the processor with instructions to perform the following processing steps: acquiring a sample image, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring a target feature based on the feature vector and the feature mean.

[0010] In this embodiment of the application, a sample image is obtained, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; a feature vector is obtained by performing feature extraction processing on the sample image using a neural network model; the mean value of the feature vector is obtained; and the target feature is obtained based on the feature vector and the mean value of the feature vector.

[0011] It is noteworthy that, in the training process, this application embodiment updates the average feature (i.e. feature mean) of a sample image with each iteration based on the momentum update mechanism. This can reduce the feature deviation error caused by the momentum factor in various optimization algorithms based on stochastic gradient descent during training. In the testing process, each sample is used to extract features through a neural network, and the extracted features minus this mean are used as the final features of the sample to complete the test.

[0012] Therefore, the embodiments of this application achieve the purpose of reducing feature deviation errors during training and testing, thereby realizing the technical effect and solving the technical problem in the prior art that the recognition accuracy of image processing algorithms is low due to the presence of occluded images and / or incomplete images. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0014] Figure 1A hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown.

[0015] Figure 2 This is a flowchart of an image processing method according to an embodiment of this application;

[0016] Figure 3 This is a schematic diagram illustrating an application scenario of an optional image processing method according to an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of this application;

[0018] Figure 5 This is a structural block diagram of another computer terminal according to an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application's solutions, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0022] Moving object re-identification: also known as moving object re-identification, is a technology that uses computer vision to determine whether a specific moving object exists in an image or video sequence. Its practical applications may include, but are not limited to: tracking lost specific moving objects in large business venues such as shopping malls, tracking suspicious specific moving objects in security venues such as banks, tracking specific moving objects violating traffic rules in areas with dense movement of moving objects such as intersections, and tracking moving specific moving objects in relatively enclosed spaces such as zoos.

[0023] Deep Neural Networks: Convolutional Neural Networks (CNNs) are a type of feedforward neural network that includes convolutional computation and has a deep structure. They are one of the representative algorithms of deep learning. CNNs have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure.

[0024] Example 1

[0025] According to an embodiment of this application, an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] The method embodiment provided in Embodiment 1 of this application can be executed in a mobile terminal, computer terminal or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned image processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0030] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0031] For the occlusion recognition problem in moving object re-identification technology, existing methods can be roughly divided into two categories: the first category is occlusion recognition methods based on pose recognition and graph matching; the second category is feature pyramid matching methods.

[0032] The first type of method typically uses keypoint detection to detect key points of moving objects. This additional input information is then used to train the model. During training, multi-layer feature matching or graph matching is needed to match images and re-identify the moving object. This type of method has two drawbacks: first, the overall performance of the model depends on the additional keypoint detection model, and its accuracy decreases as the quality of the keypoint detection model declines; second, multi-layer feature matching or graph matching is overly complex, increasing computational cost.

[0033] The second type of method generally inputs the original image directly into the model for feature extraction. During the testing phase, the features of each image are multi-scale fusion features extracted by the model. Occluded and unoccluded images are matched through feature reconstruction to re-identify moving objects. This type of method has two drawbacks: first, the reconstruction and matching of multi-scale features is computationally complex and computationally expensive; second, multi-scale feature matching, especially when occlusions are the same, is more prone to mismatches.

[0034] Moving object re-identification technology plays an important role in offline security, new retail and other businesses. In order to solve the problem that the accuracy of the algorithm model is reduced due to the occlusion and incompleteness of moving objects in offline scenarios, this application proposes a moving object re-identification method for occluded scenarios, which can improve the recognition accuracy of the model under the conditions of occluded and incomplete moving objects without increasing the model complexity (cost).

[0035] Under the aforementioned operating environment, this application provides the following: Figure 2 This illustrates an image processing method. Figure 2 This is a flowchart of an image processing method according to an embodiment of this application, such as... Figure 2 As shown, the above image processing method includes the following steps:

[0036] Step S202: Obtain sample images, wherein the sample images include: original input images and target input images, and the target input images are obtained by performing data augmentation processing on the original input images;

[0037] Step S204: Use a neural network model to perform feature extraction processing on the above sample images to obtain feature vectors;

[0038] Step S206: Obtain the feature mean corresponding to the above feature vector;

[0039] Step S208: Obtain the target features based on the above feature vectors and the above feature mean.

[0040] In this embodiment of the application, a sample image is obtained, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; a feature vector is obtained by performing feature extraction processing on the sample image using a neural network model; the mean value of the feature vector is obtained; and the target feature is obtained based on the feature vector and the mean value of the feature vector.

[0041] It is noteworthy that, in the training process, this application embodiment updates the average features of a sample image based on the momentum update mechanism in each iteration. This can reduce the feature deviation error caused by the momentum factor in various optimization algorithms based on stochastic gradient descent during training. In the testing process, each sample is used to extract features through a neural network, and the extracted features minus the mean are used as the final features of the sample to complete the test.

[0042] Therefore, the embodiments of this application achieve the purpose of reducing feature deviation errors during training and testing, thereby realizing the technical effect and solving the technical problem in the prior art that the recognition accuracy of image processing algorithms is low due to the presence of occluded images and / or incomplete images.

[0043] As an optional embodiment, the image processing method provided in this application embodiment can be applied, but is not limited to, to the analysis fields of smart retail, intelligent video surveillance, intelligent security, and smart security, for example, in visual analysis application scenarios such as security surveillance systems, large supermarkets, retail without moving objects, and large department stores.

[0044] In one optional embodiment, obtaining the target input image includes:

[0045] Step S302: The original input image is copied to obtain a copied image;

[0046] Step S304: Select the area to be occluded from the copied image above;

[0047] Step S306: The area to be occluded is erased to obtain the target input image.

[0048] like Figure 3 As shown, the image processing method provided in this application embodiment can be mainly divided into three parts, but not limited to: the first part is the data augmentation part; the second part is the network structure based on deep convolutional neural networks; and the third part is the training and testing part.

[0049] In the first part, the scheme proposed in the embodiments of this application adopts a novel data augmentation method. In each training batch, the original input image is copied to obtain a copy image, and the copy image is randomly occluded in batches to determine the area to be occluded.

[0050] For example, the copied image can be randomly divided into 6-8 parts vertically, and then one part can be randomly selected for occlusion to determine the area to be occluded. When the occlusion position is selected, all copied images in the batch can be erased at the selected position, that is, the value of the occluded area is set to 0. The purpose of data augmentation of the target input image in this embodiment is to artificially add occlusion data. The random occlusion data augmentation method based on training batches effectively solves the problem of insufficient occlusion data in the existing training set, thereby improving the training accuracy.

[0051] In one optional embodiment, the neural network model includes an attention network model, wherein the attention network model is a multilayer residual network model.

[0052] Optionally, the neural network model proposed in the embodiments of this application can be arbitrarily replaced in terms of network structure, that is, any network framework can be used as the neural network model of the solution in this application.

[0053] In one optional embodiment, the number of residual layers in the above multilayer residual network model is 50.

[0054] In an optional embodiment, a decoupled nonlocal network model is added to the first target residual layer and the second target residual layer of the above-described multilayer residual network model.

[0055] In one optional embodiment, the first target residual layer is the second residual layer of the multilayer residual network model, and a first number of the decoupled nonlocal network models are added to the second residual layer.

[0056] In one optional embodiment, the second target residual layer is the third residual layer of the multilayer residual network model, and a second number of the decoupled nonlocal network models are added to the third residual layer, wherein the second number is greater than the first number.

[0057] In the embodiments of this application, it is still as follows Figure 3 As shown, the second part is an attention network model based on a deep convolutional neural network. In this embodiment, the feature extraction architecture used in the attention network model is based on a residual network architecture, i.e., a multi-layer residual network model.

[0058] Optionally, the multilayer residual network model used in this embodiment is the 50-layer ResNet-50 residual network model. In the second and third residual layers of the multilayer residual network model, two decoupled non-local network models and three decoupled non-local network models (i.e., decoupled non-local models) are added respectively. By adding decoupled non-local network models to the multilayer residual network model (convolutional neural network model), this embodiment can significantly increase the network performance of the multilayer residual network model.

[0059] In an optional embodiment, the above-described neural network model is used to perform feature extraction processing on the above-described sample images to obtain the above-described feature vector, which includes:

[0060] Step S402: Use the above attention network model to perform feature extraction processing on the above sample image to obtain the above feature vector.

[0061] In this embodiment of the application, by adding a decoupled nonlocal network model to the multilayer residual network model, the network's ability to extract features can be enhanced.

[0062] Optionally, the specific formula for the above decoupling nonlocal network model can be, but is not limited to, the following:

[0063] w(x i x j )=σ((q i -u q ) T (k j -u k ))+(W m x j (1)

[0064] Where, x i and x j q represents two C-dimensional feature vectors in the feature map space; i and k j It is to decouple the original x in the nonlocal network model. i and x j The key and query values ​​obtained after linear transformation; u q and u k It is the average of the key value and the query value; W m It is a linear transformation.

[0065] Since the original nonlocal network model is decoupled using the above formula (1) in this embodiment, the multilayer residual network model can better respond to instances and edges in the target input image.

[0066] In an optional embodiment, the above image processing method further includes:

[0067] Step S502: During the training of the above neural network model, the classification loss function is used to constrain the classification loss of the above feature vectors.

[0068] Still Figure 3 As shown, the third part above is the training and testing section. During training, in addition to using the commonly used classification loss function, this application proposes a novel loss function based on sample pairs.

[0069] For each input sample, after passing through the classifier, the feature vector is constrained by the classification loss function L1, which can be expressed as follows:

[0070]

[0071] Where CE(.) is the cross-entropy loss, It is the true class label of the sample.

[0072] In an optional embodiment, the above image processing method further includes:

[0073] Step S602: During the training of the above neural network model, the above feature vectors are subjected to penalty loss constraints using the exponential penalty loss function.

[0074] The exponential penalty loss function is determined by the Euclidean distance between the sample image and the feature vector, and whether the sample images originate from the same target object.

[0075] Optionally, the exponential penalty loss function proposed in this application is a sample pair-based exponential penalty loss function, and the formula for the exponential penalty loss function L2 can be as follows:

[0076]

[0077] Where d is the Euclidean distance between features of a sample pair, and s represents the relationship between sample pairs.

[0078] It should be noted that when the sample pair originates from the same moving object, s=1, otherwise s=0. This can effectively shorten the distance between the sample features of the same moving object, while increasing the distance between samples of different moving objects.

[0079] The exponential penalty loss function based on sample pairs proposed in the embodiments of this application can achieve the technical effect of effectively constraining network learning.

[0080] In an optional embodiment, obtaining the mean value of the features corresponding to the feature vectors includes:

[0081] Step S702: During the training of the above neural network model, the mean value of the features corresponding to the above feature vectors is obtained based on the momentum update mechanism.

[0082] As an optional embodiment, during the training of the above-mentioned neural network model, this application embodiment also proposes a momentum update mechanism, which updates the average feature (i.e., feature mean) of a training sample in each iteration. This average feature is used during testing to reduce feature deviation errors caused by momentum factors in various optimization algorithms based on stochastic gradient descent during training.

[0083] It should be noted that, in this embodiment of the application, the average features of the training samples can be updated based on the average features of the previous time step and the feature vector obtained during training at the current time step, without imposing a constraint on minimizing the momentum error. For example, f_t = 0.9 * f_t-1 + 0.1 * f, where f_t and f_t-1 are the average features of the current time step and the previous time step step, respectively, and f is the feature vector obtained at the current time step.

[0084] In an optional embodiment, obtaining the target features based on the feature vector and the mean of the features includes:

[0085] Step S802: During the testing of the above neural network model, the above feature vector and the above feature mean are subtracted to obtain the above target feature.

[0086] As an optional embodiment, during the testing of the above neural network model, each sample undergoes feature extraction through the neural network, and the extracted feature vector minus the feature mean is used as the target feature of the sample image, thereby completing the test.

[0087] As can be seen from the above examples, compared with the first type of method in the prior art, the present application does not use any key point detection model, but is a completely end-to-end model, and does not use any graph matching method in the testing phase, thus the computational load is low.

[0088] Compared with the second type of solution in the prior art, the solution of this application does not construct a feature pyramid for feature matching, which reduces the amount of computation and reduces the false matching caused by successfully matching the same occluder.

[0089] Furthermore, in conventional moving object re-identification applications, i.e., unoccluded moving object re-identification, the proposed solution significantly outperforms existing moving object re-identification methods under all occlusion conditions. This proposed solution is more suitable for moving object re-identification applications where the presence or absence of occlusion is not explicitly known.

[0090] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0092] Example 2

[0093] According to an embodiment of this application, an apparatus embodiment for implementing the above-described image processing method is also provided. Figure 4 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of this application, as shown below. Figure 4 As shown, the device includes: a first acquisition module 400, an extraction module 402, a second acquisition module 404, and a third acquisition module 406, wherein:

[0094] The first acquisition module 400 is used to acquire sample images, wherein the sample images include: an original input image and a target input image, and the target input image is an image obtained by performing data augmentation processing on the original input image; the extraction module 402 is used to perform feature extraction processing on the sample images using a neural network model to obtain feature vectors; the second acquisition module 404 is used to acquire the feature mean corresponding to the feature vectors; and the third acquisition module 406 is used to acquire target features based on the feature vectors and the feature mean.

[0095] In this embodiment of the application, a sample image is obtained, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; a feature vector is obtained by performing feature extraction processing on the sample image using a neural network model; the mean value of the feature vector is obtained; and the target feature is obtained based on the feature vector and the mean value of the feature vector.

[0096] It is noteworthy that, in the training process, this application embodiment updates the average features of a sample image based on the momentum update mechanism in each iteration. This can reduce the feature deviation error caused by the momentum factor in various optimization algorithms based on stochastic gradient descent during training. In the testing process, each sample is used to extract features through a neural network, and the extracted features minus the mean are used as the final features of the sample to complete the test.

[0097] Therefore, the embodiments of this application achieve the purpose of reducing feature deviation errors during training and testing, thereby realizing the technical effect and solving the technical problem in the prior art that the recognition accuracy of image processing algorithms is low due to the presence of occluded images and / or incomplete images.

[0098] It should be noted that the first acquisition module 400, extraction module 402, second acquisition module 404, and third acquisition module 406 mentioned above correspond to steps S202 to S208 in Embodiment 1. The four modules and their corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0099] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Method Embodiment 1, and will not be repeated here.

[0100] Example 3

[0101] According to an embodiment of this application, an embodiment of an image processing device is also provided. This image processing device can be any computing device from a group of computing devices. The image processing device includes: a processor and a memory, wherein:

[0102] A processor; and a memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: acquiring a sample image, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring a target feature based on the feature vector and the feature mean.

[0103] In this embodiment of the application, a sample image is obtained, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; a feature vector is obtained by performing feature extraction processing on the sample image using a neural network model; the mean value of the feature vector is obtained; and the target feature is obtained based on the feature vector and the mean value of the feature vector.

[0104] It is noteworthy that, in the training process, this application embodiment updates the average features of a sample image based on the momentum update mechanism in each iteration. This can reduce the feature deviation error caused by the momentum factor in various optimization algorithms based on stochastic gradient descent during training. In the testing process, each sample is used to extract features through a neural network, and the extracted features minus the mean are used as the final features of the sample to complete the test.

[0105] Therefore, the embodiments of this application achieve the purpose of reducing feature deviation errors during training and testing, thereby realizing the technical effect and solving the technical problem in the prior art that the recognition accuracy of image processing algorithms is low due to the presence of occluded images and / or incomplete images.

[0106] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0107] Example 4

[0108] According to an embodiment of this application, an embodiment of a computer terminal is also provided. This computer terminal can be any one of a group of computer terminal devices. Optionally, in this embodiment, the aforementioned computer terminal can also be replaced with a mobile terminal or other terminal device.

[0109] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0110] In this embodiment, the computer terminal can execute the program code for the following steps in the image processing method: acquiring a sample image, wherein the sample image includes an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring target features based on the feature vector and the feature mean.

[0111] Optionally, Figure 5 This is a structural block diagram of another computer terminal according to an embodiment of this application, such as... Figure 5 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 502, memory 504, and peripheral interfaces 506.

[0112] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0113] The processor can invoke information and application programs stored in the memory through the transmission device to perform the following steps: acquiring a sample image, wherein the sample image includes an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring target features based on the feature vector and the feature mean.

[0114] Optionally, the processor may also execute program code that performs the following steps: copying the original input image to obtain a copied image; selecting the area to be occluded from the copied image; and erasing the area to be occluded to obtain the target input image.

[0115] Optionally, the processor may also execute program code that performs the following steps: using the attention network model to perform feature extraction processing on the sample image to obtain the feature vector.

[0116] Optionally, the processor may also execute program code that performs the following steps: during the training of the neural network model, the classification loss function is used to constrain the classification loss of the feature vector.

[0117] Optionally, the processor may also execute program code that performs the following steps: during the training of the neural network model, the feature vector is subjected to a penalty loss constraint using an exponential penalty loss function, wherein the exponential penalty loss function is determined by the Euclidean distance between the sample image and the feature vector and whether the sample image originates from the same target object.

[0118] Optionally, the processor may also execute program code that performs the following steps: during the training of the neural network model, the mean value of the features corresponding to the feature vectors is obtained based on the momentum update mechanism.

[0119] Optionally, the processor may also execute program code that performs the following steps: during the testing of the neural network model, subtracting the feature vector from the feature mean to obtain the target feature.

[0120] This application provides an image processing scheme. It involves acquiring a sample image, which includes an original input image and a target input image, wherein the target input image is obtained by performing data augmentation processing on the original input image; using a neural network model to perform feature extraction processing on the sample image to obtain a feature vector; obtaining the feature mean corresponding to the feature vector; and obtaining target features based on the feature vector and the feature mean.

[0121] It is noteworthy that, in the training process, this application embodiment updates the average features of a sample image based on the momentum update mechanism in each iteration. This can reduce the feature deviation error caused by the momentum factor in various optimization algorithms based on stochastic gradient descent during training. In the testing process, each sample is used to extract features through a neural network, and the extracted features minus the mean are used as the final features of the sample to complete the test.

[0122] Therefore, the embodiments of this application achieve the purpose of reducing feature deviation errors during training and testing, thereby realizing the technical effect and solving the technical problem in the prior art that the recognition accuracy of image processing algorithms is low due to the presence of occluded images and / or incomplete images.

[0123] Those skilled in the art will understand that Figure 5The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic devices. For example, a computer terminal may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0124] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0125] Example 5

[0126] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the image processing method provided in Embodiment 1 above.

[0127] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0128] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: acquiring a sample image, wherein the sample image includes: an original input image and a target input image, the target input image being an image obtained by performing data augmentation processing on the original input image; performing feature extraction processing on the sample image using a neural network model to obtain a feature vector; acquiring the feature mean corresponding to the feature vector; and acquiring target features based on the feature vector and the feature mean.

[0129] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: copying the original input image to obtain a copied image; selecting a region to be occluded from the copied image; and erasing the region to be occluded to obtain the target input image.

[0130] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: performing feature extraction processing on the sample image using the attention network model described above to obtain the feature vector described above.

[0131] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: during the training of the above neural network model, the above feature vector is subjected to classification loss constraints using a classification loss function.

[0132] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: during the training of the neural network model, the feature vector is subjected to a penalty loss constraint using an exponential penalty loss function, wherein the exponential penalty loss function is determined by the Euclidean distance between the sample image and the feature vector and whether the sample image originates from the same target object.

[0133] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: during the training of the above neural network model, the mean value of the features corresponding to the above feature vectors is obtained based on the momentum update mechanism.

[0134] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: during the testing of the above-mentioned neural network model, a subtraction operation is performed using the above-mentioned feature vector and the above-mentioned feature mean to obtain the above-mentioned target feature.

[0135] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0136] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0141] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Acquire sample images, wherein the sample images include: an original input image and a target input image, wherein the target input image is an image obtained by performing data augmentation processing on the original input image; The sample image is processed using a neural network model to extract features and obtain a feature vector. During the training of the neural network model, the mean feature value corresponding to the feature vector is obtained based on the momentum update mechanism, wherein the mean feature value corresponding to the feature vector is determined based on the feature vector obtained at the current time and the average feature value at the previous time. During the testing of the neural network model, the target features are obtained by subtracting the feature vector from the mean feature value. The process of obtaining the target input image includes: copying the original input image to obtain a copied image; selecting a region to be occluded from the copied image; and erasing the region to be occluded to obtain the target input image.

2. The image processing method according to claim 1, characterized in that, The neural network model includes an attention network model, wherein the attention network model is a multi-layer residual network model.

3. The image processing method according to claim 2, characterized in that, The number of residual layers in the multilayer residual network model is 50.

4. The image processing method according to claim 2, characterized in that, A decoupled nonlocal network model is added to the first target residual layer and the second target residual layer of the multi-layer residual network model.

5. The image processing method according to claim 4, characterized in that, The first target residual layer is the second residual layer of the multilayer residual network model, and the second residual layer contains a first number of the decoupled nonlocal network models.

6. The image processing method according to claim 5, characterized in that, The second target residual layer is the third residual layer of the multilayer residual network model, and a second number of the decoupled nonlocal network models are added to the third residual layer, wherein the second number is greater than the first number.

7. The image processing method according to any one of claims 2 to 6, characterized in that, The feature vector obtained by performing feature extraction processing on the sample image using the neural network model includes: The attention network model is used to extract features from the sample image to obtain the feature vector.

8. The image processing method according to claim 1, characterized in that, The image processing method further includes: During the training of the neural network model, a classification loss function is used to constrain the classification loss of the feature vector.

9. The image processing method according to claim 1, characterized in that, The image processing method further includes: During the training of the neural network model, the feature vector is subjected to a penalty loss constraint using an exponential penalty loss function, wherein the exponential penalty loss function is determined by the Euclidean distance between the sample image and the feature vector and whether the sample images originate from the same target object.

10. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire sample images, wherein the sample images include: an original input image and a target input image, and the target input image is an image obtained by performing data augmentation processing on the original input image; The extraction module is used to perform feature extraction processing on the sample image using a neural network model to obtain a feature vector; The second acquisition module is used to acquire the feature mean corresponding to the feature vector based on the momentum update mechanism during the training of the neural network model, wherein the feature mean corresponding to the feature vector is determined based on the feature vector acquired at the current time and the average feature of the previous time. The third acquisition module is used to perform a subtraction operation between the feature vector and the feature mean during the testing of the neural network model to obtain the target feature; The first acquisition module is further configured to: copy the original input image to obtain a copied image; select a region to be occluded from the copied image; and erase the region to be occluded to obtain the target input image.

11. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the image processing method according to any one of claims 1 to 9.

12. An image processing device, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Acquire sample images, wherein the sample images include: an original input image and a target input image, wherein the target input image is an image obtained by performing data augmentation processing on the original input image; The sample image is processed using a neural network model to extract features and obtain a feature vector. During the training of the neural network model, the mean feature value corresponding to the feature vector is obtained based on the momentum update mechanism, wherein the mean feature value corresponding to the feature vector is determined based on the feature vector obtained at the current time and the average feature value at the previous time. During the testing of the neural network model, the target features are obtained by subtracting the feature vector from the mean feature value. The process of obtaining the target input image includes: copying the original input image to obtain a copied image; selecting a region to be occluded from the copied image; and erasing the region to be occluded to obtain the target input image.

Images