Image processing method and apparatus, device, and medium

Abstract

The present application discloses an image processing method and apparatus, a device, and a medium. The method comprises: acquiring a first sample image comprising a plurality of regional images; calling an attention mechanism network to identify a first regional image and a second regional image with different attention degrees among the plurality of regional images on the basis of an attention parameter; adding a first predicted label to the first regional image, and adding a second predicted label to the second regional image, wherein the first predicted label and the second predicted label are used for indicating different image definition; acquiring a first reference label of the first regional image and a second reference label of the second regional image; and correcting the attention parameter on the basis of the difference between the first predicted label and the first reference label and the difference between the second predicted label and the second reference label, wherein the corrected attention parameter is used for extracting an embedded feature for classification of a palmprint image. Use of the present application can improve the accuracy of extraction of embedded features of palmprint images and the accuracy of classification of palmprint images.

Description

Image processing method, device, equipment and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 25, 2023, with application number 2023110802296 and application name “Image processing method, device, equipment and medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of artificial intelligence, and in particular to an image processing method, apparatus, device and medium. Background Art

[0003] In the scenario of palmprint image classification, the palmprint image is usually first embedded through a feature extraction network to extract the embedded features of the palmprint image, and then the palmprint image is classified using the embedded features.

[0004] Palmprint images are captured by a camera and may contain several regions of varying clarity. In existing applications, palmprint images can be directly embedded using a feature extraction network. However, during this process, the feature extraction network may overemphasize unclear regions within the palmprint image. These regions often have inaccurate features, leading to inaccurate embedded features and, consequently, inaccurate palmprint image classification results.

[0005] Summary of the Invention

[0006] The present application provides an image processing method, apparatus, device and medium, which can improve the accuracy of extracting embedded features of palmprint images, thereby improving the accuracy of palmprint image classification processing.

[0007] On one hand, the present application provides an image processing method, the method comprising:

[0008] Acquire a first sample image, wherein the first sample image includes a plurality of divided regional images, and if the regional images have different sharpnesses, the regional images belong to different sharpness types;

[0009] Invoking an attention mechanism network to perform attention degree recognition processing on the multiple region images to obtain a first region image and a second region image from the multiple region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter;

[0010] Based on the attention mechanism network's respective attention levels to the first region image and the second region image, a first prediction label is added to the first region image, and a second prediction label is added to the second region image, where the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, where the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type;

[0011] Acquire a first reference tag of the first region image and a second reference tag of the second region image, wherein the first reference tag is used to indicate that the first region image belongs to a first reference definition type, and the second reference tag is used to indicate that the second region image belongs to a second reference definition type;

[0012] Based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label, the attention parameter is modified, and the attention mechanism network is used to extract the embedded features of the palmprint image based on the modified attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

[0013] In one aspect, the present application provides an image processing device, comprising:

[0014] A first acquisition module is used to acquire a first sample image, wherein the first sample image includes a plurality of divided regional images, and if the regional images have different sharpnesses, the regional images belong to different sharpness types;

[0015] a calling module, configured to call an attention mechanism network to perform attention degree recognition processing on the multiple region images to obtain a first region image and a second region image from the multiple region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter;

[0016] an adding module, configured to add a first prediction label to the first region image and a second prediction label to the second region image based on the attention of the attention mechanism network to each of the first region image and the second region image, wherein the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, and the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type;

[0017] a second acquisition module, configured to acquire a first reference label of the first region image and a second reference label of the second region image, wherein the first reference label is used to indicate that the first region image belongs to a first reference definition type, and the second reference label is used to indicate that the second region image belongs to a second reference definition type;

[0018] The correction module is used to correct the attention parameter based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label. The attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

[0019] In one aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor executes the method in one aspect of the present application.

[0020] In one aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0021] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in various optional embodiments such as the above-mentioned aspect.

[0022] The first sample image of the present application may include multiple divided regional images. For the multiple regional images, if the clarity of the regional images is different, the clarity types of the regional images may be different. In other words, the present application may divide and process the first sample image according to the different clarity of the images of each part of the first sample image. Thus, the present application may call the attention mechanism network to perform attention recognition processing on the multiple regional images to obtain the first regional image and the second regional image in the multiple regional images; wherein the attention mechanism network includes attention parameters, and the attention mechanism network may determine that the attention to the first regional image is higher than the attention to the second regional image through the attention parameters; thus, the present application may add a first prediction label to the first regional image and a second prediction label to the second regional image based on the attention of the attention mechanism network to the respective attention of the first regional image and the second regional image; the first prediction label is used to indicate that the first regional image belongs to the first predicted clarity type; the second prediction label is used to indicate that the second regional image belongs to the second predicted clarity type; the clarity indicated by the first predicted clarity type is higher than the clarity indicated by the second predicted clarity type. clarity; that is, the present application can mark the area image that the attention mechanism network pays more attention to with a predicted label with a higher image clarity; in addition, the present application can also obtain a first reference label for the first area image and a second reference label for the second area image; the first reference label is used to indicate the actual first reference clarity type to which the first area image belongs; the second reference label is used to indicate the actual second reference clarity type to which the second area image belongs; thus, the present application can correct the attention parameters of the attention mechanism network based on the difference between the first predicted label and the first reference label, as well as the difference between the second predicted label and the second reference label; subsequently, the attention mechanism network can extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features can be used to identify the owner of the palmprint in the palmprint image.It can be seen that the method proposed in the present application can add prediction labels to the regional images of the first sample image by paying attention to the regional images in the first sample image through the attention mechanism network, and the clarity of the clarity type indicated by the prediction label added to the regional image with higher attention can be higher. Then, the attention parameter can be corrected by the difference between the real label of the regional image (such as the reference label) and the predicted label. Subsequently, the attention mechanism network can use the corrected attention parameter to pay higher attention to the regional images with higher clarity in the palmprint image, and pay lower attention to the regional images with lower clarity in the palmprint image. In this way, the image features of the regional images with higher clarity in the palmprint image can be extracted to the greatest extent, and finally more accurate embedding features of the palmprint image can be extracted. Through the more accurate embedding features of the palmprint image, more accurate classification processing of the palmprint image can be achieved (that is, more accurate identification processing of the owner of the palmprint in the palmprint image). BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a schematic diagram of a network architecture for palm image processing provided by an embodiment of the present application;

[0024] FIG2 is a schematic diagram of a scenario for correcting network parameters provided in an embodiment of the present application;

[0025] FIG3 is a flow chart of an image processing method provided in an embodiment of the present application;

[0026] FIG4 is a schematic diagram of a process for training an image segmentation network according to an embodiment of the present application;

[0027] FIG5 is a schematic diagram of a scenario for correcting attention parameters provided by an embodiment of the present application;

[0028] FIG6 is a flow chart of a network training method provided in an embodiment of the present application;

[0029] FIG7 is a schematic diagram of a scenario for training a palmprint classification network provided in an embodiment of the present application;

[0030] FIG8 is a schematic diagram of a scenario for generating target embedding features provided by an embodiment of the present application;

[0031] FIG9 is a schematic diagram of a palmprint prediction scenario provided by an embodiment of the present application;

[0032] FIG10 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0033] FIG11 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] This application relates to technologies related to artificial intelligence. Artificial Intelligence (AI) refers to theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is the study of the design principles and implementation methods of various intelligent machines, enabling them to have the capabilities of perception, reasoning, and decision-making.

[0035] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0036] This application mainly involves machine learning in artificial intelligence. Among them, machine learning (ML) is a multi-disciplinary interdisciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by formula.

[0037] The machine learning involved in this application mainly refers to how to train a palmprint classification network and then accurately classify the palmprint categories of palmprint images through the trained palmprint classification network. The specific process can be found in the relevant description of the embodiment corresponding to Figure 3 below.

[0038] First of all, it should be noted that all data collected in this application (such as all related data such as palmprint images) are collected with the consent and authorization of the object to which the data belongs (such as users, institutions or enterprises), and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0039] Please refer to Figure 1, which is a structural diagram of a network architecture for palm image processing provided by an embodiment of the present application. As shown in Figure 1, the network architecture may include a server 200 and a terminal device cluster. The terminal device cluster may include one or more terminal devices, and the number of terminal devices will not be restricted here. As shown in Figure 1, the multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n; as shown in Figure 1, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all be connected to the server 200 through a network connection, so that each terminal device can exchange data with the server 200 through a network connection.

[0040] The server 200 shown in FIG1 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smart terminal such as a smartphone, tablet computer, laptop computer, desktop computer, smart TV, or in-vehicle terminal. The following describes the embodiment of the present application in detail, taking the communication between the terminal device 1 and the server 200 as an example.

[0041] Please also refer to Figure 2, which is a schematic diagram of a scenario for modifying network parameters provided by an embodiment of the present application. As shown in Figure 2, the server 200 can modify the attention parameters of the attention mechanism network based on a first sample image. The first sample image can include multiple region images (partial images belonging to the first sample image). The attention mechanism network can determine the degree of attention to each region image in the input image (such as the first sample image) based on the attention parameters.

[0042] Therefore, the server 200 can add prediction labels to each area image in the first sample image through the attention parameters of the attention mechanism network. The prediction labels can be labels of the clarity types of each area image in the first sample image determined by the attention parameters of the attention mechanism network. For example, the area images that the attention mechanism network pays more attention to can be labeled with a clarity type with a higher clarity.

[0043] The server 200 may also obtain reference tags of the respective region images in the first sample image. The reference tags may be used to indicate the actual definition type of the respective region images in the first sample image.

[0044] Therefore, the server 200 can correct the attention parameters of the attention mechanism network based on the difference between the predicted labels and the reference labels of the images of each area in the first sample image, so that the attention mechanism network can pay more attention to the features of the images in clearer areas in the input image through the corrected attention parameters.

[0045] Subsequently, the attention mechanism network can be used as a network for feature extraction of palmprint images in the palmprint recognition scenario. The attention mechanism network can embed the input palmprint image through the attention parameters modified in the above manner to generate accurate embedding features of the palmprint image, and then accurately classify the palmprint category in the palmprint image through the embedded features. The palmprint image can be an image obtained by the terminal device 1 taking a photo of the user's palmprint. The terminal device 1 can send the collected palmprint image to the server 200 to request the server 200 to classify the palmprint category of the palmprint image, so that the server 200 can call the attention mechanism network to classify the palmprint category of the palmprint image through the modified attention parameters. The above specific process can also be found in the relevant description of the corresponding embodiments of Figures 3 and 6 below.

[0046] By adopting the method of the present application, the attention mechanism network can focus more on the features of the image in the clearer area of ​​the input palmprint image, thereby extracting more accurate embedding features of the palmprint image. The accurate embedding features of the palmprint image can also achieve more accurate classification processing of the palmprint category of the palmprint image.

[0047] Please refer to Figure 3, which is a flow chart of an image processing method provided in an embodiment of the present application. The execution subject in the embodiment of the present application can be an image processing device, which can be a computer device or a computer device cluster composed of multiple computer devices. The computer device can be a server, a terminal device, or other devices, without limitation. As shown in Figure 3, the method may include:

[0048] Step S101 : obtaining a first sample image. The first sample image includes a plurality of divided regional images. If the regional images have different sharpnesses, the regional images belong to different sharpness types.

[0049] Optionally, the image processing device may acquire a first sample image, which may include multiple divided regional images. In other words, the first sample image may be divided into the multiple regional images, each of which may be a local image within the first sample image. Specifically, the first sample image may be divided into multiple image blocks (i.e., multiple regional images), the size of each image block may be determined based on the actual application scenario (may be a pre-set division size), and the sizes of the image blocks may be the same.

[0050] The multiple regional images may include regional images with the same clarity, or regional images with different clarity (i.e., image clarity). The clarity of an image may refer to the sharpness of changes in the edges of image details, that is, the clarity of image details and their boundaries. In this application, if the clarity of the regional images is different, it can be considered that the regional images belong to different clarity types. The clarity type of an image may be understood as the clarity level of the image, and the clarity level may refer to the clarity ranges obtained by dividing the clarity of the image, and one clarity level may correspond to one clarity range. Each clarity type of an image may be used to indicate its corresponding clarity (image clarity), such as to indicate its corresponding clarity level. The higher the clarity level of an image, the higher the clarity indicated for the image (i.e., the clearer the image), and conversely, the lower the clarity level of an image, the lower the clarity indicated for the image (i.e., the less clear the image).

[0051] For example, the clarity types of an image may include at least two types, which may include a blurred clarity type and a clear (or high-definition) clarity type. As the name suggests, the clarity indicated by the blurred clarity type (blurred clarity) is lower than the clarity indicated by the clear clarity type (clear clarity). Alternatively, on this basis, it may be divided into the above-mentioned finer clarity types according to the degree of clarity. For example, the clarity type of an image may include a blurred (such as low-definition) clarity type, a standard-definition clarity type, a high-definition clarity type, an ultra-high-definition clarity type, and so on. The clarity indicated by each clarity type may be increasing in sequence. For example, the clarity indicated by the standard-definition clarity type is higher than the clarity indicated by the blurred clarity type, the clarity indicated by the high-definition clarity type is higher than the clarity indicated by the standard-definition clarity type, and the clarity indicated by the ultra-high-definition clarity type is higher than the clarity indicated by the high-definition clarity type.

[0052] The types of image clarity and the specific types of clarity included can be arbitrarily set according to actual application scenarios and are not limited thereto. It is understood that different clarity of regional images may refer to different clarity levels of regional images.

[0053] In other words, the first sample images may include regional images of at least two definition types, and one regional image may have one definition type. The specific number of the first sample images may be determined according to actual application scenarios.

[0054] Step S102: Call the attention mechanism network to perform attention recognition processing on multiple region images to obtain a first region image and a second region image among the multiple region images. The attention mechanism network includes an attention parameter. The attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter.

[0055] Optionally, the present application may allow the attention mechanism network to perform attention recognition processing on the input image (such as the first sample image) according to the various region images divided into the input image. Therefore, the image processing device may call the attention mechanism network to perform recognition processing (such as attention recognition processing) on ​​multiple region images of the above-mentioned first sample image, thereby obtaining the first region image and the second region image among the multiple region images.

[0056] Among them, the attention mechanism network is a network (i.e., a model) with an attention mechanism. The core idea of ​​the attention mechanism is to simulate the human attention process on the network input, that is, the network can automatically determine which parts of the input data need to be paid attention to when processing data, so that the network can effectively capture the key information of the input data. Exemplarily, the attention mechanism network includes but is not limited to the Transformer (a neural network based on the self-attention mechanism), GAT (a graph attention mechanism network), etc. The attention mechanism network can include attention parameters (which are weight parameters and are network parameters of the attention mechanism network). The attention mechanism network can use the attention parameters to determine that the attention to the first region image is higher than the attention to the second region image (i.e., the degree of attention), that is, the attention mechanism network pays more attention to the first region image than to the second region image. The attention mechanism network's attention to each region image can be determined by the attention parameters of the attention mechanism network, as described below.

[0057] The attention mechanism network can use the above-mentioned attention parameters to perform recognition processing (i.e., feature learning) on ​​the first sample image according to each regional image of the first sample image, so as to identify (i.e., learn) the attention weight of the attention mechanism network for each regional image. The attention mechanism network can have an attention weight for a regional image. The attention weight of the attention mechanism network for a regional image can be used to reflect the attention of the attention mechanism network to the regional image. The higher the attention weight, the higher the attention degree. Conversely, the lower the attention weight, the lower the attention degree. The value range of the attention weight can be [0, 1].

[0058] As can be seen from the above, the attention mechanism network can obtain the attention weight for each image block of the first sample image (i.e., each region image) through the attention parameter. In other words, the attention parameter of the attention mechanism network can be used to know which image blocks of the input image (such as the first sample image) the attention mechanism network pays more attention to and which image blocks of the input image it pays less attention to.

[0059] Optionally, the value range [0, 1] of the attention weight can be divided according to the number of image clarity types to obtain multiple weight ranges of the attention weight. One clarity type can correspond to a weight range of the attention weight. The higher the clarity indicated by the clarity type, the higher the weight value in the weight range corresponding to the clarity type. All weight ranges combined can be the entire value range [0, 1] of the attention weight.

[0060] For example, if there are two types of image clarity, including a blurred clarity type and a clear clarity type, the value range of the attention weight can be divided into two weight ranges, such as the weight range [0, 0.5) and the weight range [0.5, 1]. The weight range [0, 0.5) can be the weight range corresponding to the blurred clarity type, and the weight range [0.5, 1] ​​can be the weight range corresponding to the clear clarity type. There is no overlap between the weight ranges.

[0061] Therefore, the attention weight of the first area image and the attention weight of the second area image can be respectively within the two divided weight ranges, and the attention weight of the first area image is greater than the attention weight of the second area image, that is, the weight value within the weight range to which the attention weight of the first area image belongs is greater than the weight value within the weight range to which the attention weight of the second area image belongs.

[0062] Among them, any two weight ranges among the multiple weight ranges into which the value range of the attention weight is divided can be referred to as the first weight range and the second weight range, and the weight value in the first weight range is greater than the weight value in the second weight range. For example, the weight range [0.5, 1] ​​can be the first weight range, and the weight range [0, 0.5) can be the second weight range.

[0063] Therefore, the present application may use, among the multiple region images of the first sample image, a region image whose weight of interest is within a first weight range as the first region image, and may use, among the multiple region images, a region image whose weight of interest is within a second weight range as the second region image. The number of the first region images and the second region images may be one or more, and the specific number of the first region images and the second region images may be determined based on the actual application scenario.

[0064] It is understandable that the present application is illustrated by taking two types of clarity types of regional images (i.e., the first regional image and the second regional image) as an example. In fact, regional images of all clarity types can (or need to) be processed in the same way to correct the attention parameters of the attention mechanism network. For example, when there are more than two types of clarity in an image, there will also be more than two weight ranges. The attention weight of the attention mechanism network for the first regional image and the attention weight of the attention mechanism network for the second regional image can be in different weight ranges respectively. The first regional image and the second regional image can be regional images whose respective attention weights are in any two weight ranges of the more than two weight ranges, and the weight value of the attention mechanism network's attention weight for the first regional image within the weight range is greater than the weight value of the attention mechanism network's attention weight for the second regional image within the weight range. In other words, for regional images whose attention weights are within the pairwise weight ranges of the more than two weight ranges, they can be respectively used as the corresponding first regional image and the second regional image to perform the relevant processing of the embodiment of the present application.

[0065] For example, if there are three types of image clarity, including blurred clarity type, standard definition clarity type and clear clarity type, the value range of the attention weight can be divided into three weight ranges, such as the weight range [0, 0.33), the weight range [0.33, 0.66) and the weight range [0.66, 1]. The weight range [0, 0.33) can correspond to the blurred clarity type, the weight range [0.33, 0.66) can correspond to the standard definition clarity type, and the weight range [0.66, 1] can correspond to the clear clarity type.

[0066] For regional images whose attention weights are within the weight range [0, 0.33) and regional images whose attention weights are within the weight range [0.33, 0.66), the regional images whose attention weights are within the weight range [0, 0.33) can be used as the second regional images, and the regional images whose attention weights are within the weight range [0.33, 0.66) can be used as the first regional images, so as to perform the relevant processing described in the embodiments of the present application.

[0067] As for the regional images whose attention weights are within the weight range [0.33, 0.66) and the regional images whose attention weights are within the weight range [0.66, 1], the regional images whose attention weights are within the weight range [0.33, 0.66) can be used as the second regional images, and the regional images whose attention weights are within the weight range [0.66, 1] can be used as the first regional images, so as to perform the relevant processing described in the embodiments of the present application.

[0068] Furthermore, for the regional image whose attention weight is within the weight range [0, 0.33) and the regional image whose attention weight is within the weight range [0.66, 1], the regional image whose attention weight is within the weight range [0, 0.33) can be used as the second regional image, and the regional image whose attention weight is within the weight range [0.66, 1] can be used as the first regional image, so as to perform the relevant processing described in the embodiments of the present application.

[0069] Among them, when there are more than three types of image clarity, corresponding processing can also be performed according to the above principles. The specific number of image clarity types and the number of weight ranges that need to be divided can be determined according to the actual application scenario, and this application does not impose any restrictions on this.

[0070] Step S103: Based on the attention mechanism network's respective attention levels to the first region image and the second region image, a first prediction label is added to the first region image, and a second prediction label is added to the second region image. The first prediction label is used to indicate that the first region image belongs to a first prediction clarity type, and the second prediction label is used to indicate that the second region image belongs to a second prediction clarity type. The clarity indicated by the first prediction clarity type is higher than the clarity indicated by the second prediction clarity type.

[0071] Optionally, the image processing device can add corresponding prediction labels to each regional image based on the weight range to which the attention weight of each regional image belongs and the clarity type corresponding to each weight range through the attention mechanism network, as described below.

[0072] The image processing device can add a first prediction label to the first area image through the attention mechanism network's attention weight (i.e., attention degree): for example, the image processing device can add a first prediction label to the first area image through the clarity type corresponding to the weight range to which the attention weight of the first area image belongs. The first prediction label can be used to indicate that the clarity type of the first area image belongs to the first predicted clarity type, and the first predicted clarity type is also the clarity type corresponding to the weight range to which the attention weight of the first area image belongs. The first predicted clarity type can be understood as the clarity type determined (i.e., predicted) by the attention mechanism network for the first area image based on the attention parameter.

[0073] Similarly, the image processing device can add a second prediction label to the second area image through the attention mechanism network's attention weight (i.e., attention degree): for example, the image processing device can add a second prediction label to the second area image through the clarity type corresponding to the weight range to which the attention weight of the second area image belongs. The second prediction label can be used to indicate that the clarity type of the second area image belongs to the second predicted clarity type, which is also the clarity type corresponding to the weight range to which the attention weight of the second area image belongs. The second predicted clarity type can be understood as the clarity type determined (i.e., predicted) by the attention mechanism network for the second area image based on the attention parameter.

[0074] Furthermore, it can be understood that the image definition indicated by the first prediction definition type is higher than the image definition indicated by the second prediction definition type.

[0075] Through the above process of adding prediction labels (including the first prediction label and the second prediction label) to the first region image and the second region image, it can be understood that the present application can add prediction labels of a higher clarity type for the region image that the attention mechanism network pays more attention to. Subsequently, by correcting the attention parameters of the attention mechanism network in this way, the attention mechanism network can also pay more attention to the clearer region images in the input image and pay less attention to the blurrier region images in the input image through the corrected attention parameters.

[0076] Step S104 : Acquire a first reference tag of the first region image and a second reference tag of the second region image, wherein the first reference tag is used to indicate that the first region image belongs to a first reference definition type, and the second reference tag is used to indicate that the second region image belongs to a second reference definition type.

[0077] Optionally, the image processing device may obtain a reference tag of the first region image and a reference tag of the second region image. The reference tag of the first region image may be referred to as a first reference tag, and the reference tag of the second region image may be referred to as a second reference tag.

[0078] The first reference label may be a label of an actual definition type of the first region image, and the second reference label may be a label of an actual definition type of the second region image.

[0079] The first reference tag can be used to indicate that the definition type of the image in the first region belongs to the first reference definition type, i.e., the actual definition type of the image in the first region can be the first reference definition type. Similarly, the second reference tag can be used to indicate that the definition type of the image in the second region belongs to the second reference definition type, i.e., the actual definition type of the image in the second region can be the second reference definition type.

[0080] The first reference definition type and the second reference definition type may be any one of the at least two definition types set for the image.

[0081] Optionally, the first reference label of the first region image and the second reference label of the second region image may be obtained through a trained clarity classification network, as described below:

[0082] The image processing device can call a trained clarity classification network to perform classification and prediction processing on the clarity of the first area image, and obtain a first reference clarity type of the first area image. The first reference clarity type is the actual clarity type of the first area image predicted by the trained clarity classification network.

[0083] Similarly, the image processing device can call the trained clarity classification network to perform classification and prediction processing on the clarity of the second area image, and obtain the above-mentioned second reference clarity type of the second area image. The second reference clarity type is the actual clarity type of the second area image predicted by the trained clarity classification network.

[0084] Therefore, the image processing device may add the first reference tag to the first region image according to the first reference definition type. The first reference tag is used to indicate that the definition type of the first region image is the first reference definition type.

[0085] The image processing device may further add the second reference tag to the second region image according to the second reference definition type. The second reference tag is used to indicate that the definition type of the second region image is the second reference definition type.

[0086] For example, the process of training the above-trained definition classification network may include: the image classification network may obtain a third sample image and the definition classification network to be trained, and the third sample image may have a definition label, and the definition label may be used to indicate the true definition type of the third sample image (i.e., the actual definition type of the third sample image). Optionally, the present application may have multiple third sample images, and the multiple third sample images may include sample images of various definition types (including all definition types pre-configured for the image) for training the definition classification network to be trained.

[0087] Furthermore, the image processing device can call the clarity classification network that needs to be trained to perform classification and prediction processing on the clarity type of the above-mentioned third sample image, and obtain the predicted clarity type of the third sample image. The predicted clarity type is the clarity type predicted for the third sample image using the clarity classification network that needs to be trained.

[0088] Then, the image processing device can correct the network parameters of the clarity classification network to be trained by the difference between the actual clarity type and the predicted clarity type of the third sample image (which can be reflected by the cross entropy loss between the actual clarity type and the predicted clarity type), and thus obtain a trained clarity classification network.

[0089] Furthermore, the image processing device may also call a trained image segmentation network to obtain a first reference label of the first region image and a second reference label of the second region image. This process may include:

[0090] The trained image segmentation network is a trained network that can be used to segment the input image into partial images of different clarity types. Exemplarily, the trained image segmentation network may include, but is not limited to, a U-Net (a convolutional neural network). Therefore, the image processing device can call the trained image segmentation network to perform image segmentation processing on the first sample image to obtain multiple segmented images of the first sample image, each of which can be a local image in the first sample image. A segmented image can correspond to a clarity type, that is, a segmented image has a clarity type. The clarity type corresponding to each segmented image can also be identified by the trained image segmentation network when segmenting each segmented image.

[0091] Furthermore, the image processing device can use the segmented image with the highest overlap with the first region image (i.e., the largest image overlap area) among the multiple segmented images as the first clarity matching image of the first region image. Furthermore, the image processing device can add a first reference label to the first region image based on the clarity type corresponding to the first clarity matching image. The first reference clarity type indicated by the first reference label can be the clarity type corresponding to the first clarity matching image.

[0092] Similarly, the image processing device can use the segmented image with the highest overlap with the second area image (i.e., the largest image overlap area) among the multiple segmented images as the second clarity matching image of the second area image. Then, the second reference label can be added to the second area image based on the clarity type corresponding to the second clarity matching image. The second reference clarity type indicated by the second reference label can be the clarity type corresponding to the second clarity matching image.

[0093] Optionally, the image processing device can adopt any of the two methods mentioned above (one is a method based on a trained clarity classification network, and the other is a method based on a trained image segmentation network) to pre-add corresponding reference labels to each area image in the first sample image, and then in this step S104, the reference labels (including the first reference label and the second reference label) pre-added to the first area image and the second area image can be directly obtained.

[0094] Please refer to Figure 4, which is a schematic diagram of a process for training an image segmentation network according to an embodiment of the present application. As shown in Figure 4, the process may include:

[0095] 1. Assume that the clarity type set for the image includes a clear clarity type and a blurred clarity type. Therefore, the image processing device can prepare a sample image containing a clear area label and a blurred area label. The clear area label can be a label given to the image of the clear area in the sample image, and the clear area label is a label of the clear clarity type; the blurred area label can be a label given to the image of the blurred area in the sample image, and the blurred area label is a label of the blurred clarity type.

[0096] 2. The image processing device can construct an image segmentation network, such as the image processing device can construct a U-Net network as an image segmentation network.

[0097] 3. The image processing device can define a loss function for the image segmentation network. For example, the loss function can be the cross entropy loss between the actual clear area and blurred area indicated by the region labels (including the above-mentioned clear area labels and blurred area labels) given to the sample image and the predicted clear area and blurred area.

[0098] 4. The image processing device can initialize the network parameters of the image segmentation network, such as performing random initialization, and then start training the image segmentation network.

[0099] 5. The image processing device can obtain samples for training the image segmentation network in the current round (belonging to the sample images prepared in step 1 above).

[0100] 6. The image processing device can input the acquired samples into the image segmentation network for forward propagation to predict and identify the clear areas and blurred areas in the input sample image.

[0101] 7. The image processing device can calculate the loss function of the image segmentation network by the difference between the predicted clear area and the actual clear area marked by the clear area label (which can be reflected by the cross entropy loss), and the difference between the predicted fuzzy area and the actual fuzzy area marked by the fuzzy area label (which can be reflected by the cross entropy loss).

[0102] 8. The image processing device can perform back propagation in the image segmentation network through the calculated loss function to correct (i.e., update) the network parameters of the image segmentation network.

[0103] 9. The image processing device may determine whether a stopping condition has been met. The stopping condition may be a condition for determining whether the image segmentation network has been trained. For example, the stopping condition may be that the network parameters of the image segmentation network have been trained to a convergence state, or the stopping condition may be that the number of rounds of iterative training of the image segmentation network is greater than or equal to a threshold number, etc. Therefore, if the stopping condition is met, the image processing device may execute step 10 below. If the stopping condition is not met, the image processing device may repeat step 5 above to begin the next round of iterative training of the image segmentation network.

[0104] 10. The image segmentation network trained at this time is used as the trained image segmentation network.

[0105] In step S105, based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label, the attention parameter is corrected, and the attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters. The embedded features are used to identify the owner of the palmprint in the palmprint image.

[0106] Optionally, the image processing device may modify the attention parameters of the attention mechanism network based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label. The difference between the first predicted label and the first reference label may be reflected by a cross-entropy loss between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label may be reflected by a cross-entropy loss between the second predicted label and the second reference label.

[0107] In other words, the attention parameters of the attention mechanism network can be corrected by the cross-entropy loss between the first predicted label and the first reference label, and the cross-entropy loss between the second predicted label and the second reference label (such as the sum of the two cross entropies). The purpose of the correction can be to make the sum of the cross entropies tend to a minimum value (such as 0).

[0108] Please refer to Figure 5, which is a schematic diagram of a scenario for modifying attention parameters provided by an embodiment of the present application. As shown in Figure 5, the first region image can have a first predicted label and a first reference label, and the second region image can have a second predicted label and a second reference label.

[0109] The image processing device can obtain the difference between the first predicted label and the first reference label of the first region image. For example, the difference can be the cross-entropy loss between the first predicted label and the first reference label. In other words, the cross-entropy loss can be used to reflect the difference between the first predicted label and the first reference label. The image processing device can also obtain the difference between the second predicted label and the second reference label of the second region image. For example, the difference can be the cross-entropy loss between the second predicted label and the second reference label. In other words, the cross-entropy loss can be used to reflect the difference between the second predicted label and the second reference label.

[0110] Furthermore, the image processing device can correct the attention parameters of the attention mechanism network by using the cross-entropy loss between the first predicted label and the first reference label, and the cross-entropy loss between the second predicted label and the second reference label obtained above to obtain the corrected attention parameters.

[0111] According to the above principle, the attention mechanism network can pay more attention to the area image with higher clarity in the input image (such as the palmprint image) through the modified attention parameters, and pay less attention to the area image with lower clarity in the input image. Therefore, subsequently, the attention mechanism network can use the modified attention parameters to more accurately extract the embedded features of the palmprint image. The accurate embedded features can be used to more accurately classify the palmprint category of the palmprint image (such as the owner of the palmprint in the palmprint image). Among them, the process of classifying the palmprint category of the palmprint image is the process of identifying the owner of the palmprint in the palmprint image (such as identifying which user the palmprint in the palmprint image specifically belongs to, so as to determine the identity of the owner of the palmprint in the palmprint image). The specific process can also be referred to the relevant description in the embodiment corresponding to Figure 6 below.

[0112] The present application can obtain a first sample image; the first sample image includes a plurality of divided regional images, and if the clarity of the regional images is different, the clarity types of the regional images are different; and the attention mechanism network can be called to perform attention recognition processing on the plurality of regional images to obtain a first regional image and a second regional image in the plurality of regional images; the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention to the first regional image is higher than the attention to the second regional image based on the attention parameter; furthermore, based on the attention of the attention mechanism network to the respective attention of the first regional image and the second regional image, a first prediction label can be added to the first regional image, and a second prediction label can be added to the second regional image; the first prediction label is used to indicate that the first regional image belongs to the first predicted clarity type; the second prediction label is used to indicate that the second area image belongs to the second prediction clarity type; the clarity indicated by the first prediction clarity type is higher than the clarity indicated by the second prediction clarity type; a first reference label of the first area image and a second reference label of the second area image can also be obtained; the first reference label is used to indicate that the first area image belongs to the first reference clarity type; the second reference label is used to indicate that the second area image belongs to the second reference clarity type; thereby, the attention parameter can be corrected based on the difference between the first prediction label and the first reference label, and the difference between the second prediction label and the second reference label; the attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image. It can be seen that the method proposed in the present application can add prediction labels to the regional images of the first sample image by paying attention to the regional images in the first sample image through the attention mechanism network, and the clarity of the clarity type indicated by the prediction label added to the regional image with higher attention can be higher. Then, the attention parameter can be corrected by the difference between the real label of the regional image (such as the reference label) and the predicted label. Subsequently, the attention mechanism network can use the corrected attention parameter to pay higher attention to the regional images with higher clarity in the palmprint image, and pay lower attention to the regional images with lower clarity in the palmprint image. In this way, the image features of the regional images with higher clarity in the palmprint image can be extracted to the greatest extent, and finally more accurate embedding features of the palmprint image can be extracted. Through the more accurate embedding features of the palmprint image, more accurate classification processing of the palmprint image can be achieved (that is, more accurate identification processing of the owner of the palmprint in the palmprint image).

[0113] Please refer to Figure 6, which is a flow chart of a network training method provided by an embodiment of the present application. The present application can be applied to the palmprint recognition scenario. As shown in Figure 6, the method may include:

[0114] Step S201: Acquire a second sample image. The second sample image is an image obtained by photographing the sample palmprint. The second sample image has a palmprint label. The palmprint label is used to indicate the true category of the sample palmprint.

[0115] Optionally, the image processing device can also obtain a second sample image, which can be an image obtained by photographing a sample palm print (which can be the palm print of any user). The second sample image can have a palm print label, which can be used to indicate the true category of the sample palm print, and the true category can be used to indicate the actual owner of the sample palm print (such as the actual user, that is, which user's palm print it is).

[0116] The above-mentioned attention mechanism network can be included in the palmprint classification network. The attention parameters (i.e., the corrected attention parameters) obtained after the attention parameters of the attention mechanism network are corrected through the process described in the embodiment corresponding to FIG3 can be frozen in the palmprint classification network, so that the corrected attention parameters do not need to be corrected during the subsequent training of the palmprint classification network. The palmprint classification network can be a network for identifying (i.e., classifying) palmprints in a palmprint image to identify the owner of the palmprint in the palmprint image (e.g., the user to whom the palmprint belongs). Exemplarily, the palmprint classification network includes but is not limited to a decision tree model, a support vector machine model, and the like.

[0117] In other words, the process in the embodiment corresponding to Figure 3 above can be understood as a pre-operation for correcting the attention parameters of the attention mechanism network in the palmprint classification network. After correcting the attention parameters of the attention mechanism network in the palmprint classification network, the palmprint classification network can also be trained as a whole. During this training process, the corrected attention parameters may not be updated, but other network parameters in the palmprint classification network except the corrected attention parameters will be updated, as described below.

[0118] In step S202, the attention mechanism network is called to perform embedding processing on the second sample image based on the corrected attention parameters to generate a sample embedding feature of the second sample image.

[0119] Optionally, the above-mentioned attention mechanism network may belong to a subnetwork in the palmprint classification network for extracting features from the input image. In addition to the corrected attention parameters, the network parameters of the attention mechanism network may also include other network parameters for embedding processing (i.e., feature extraction) of the input image.

[0120] Therefore, the image processing device can call the attention mechanism network in the palmprint classification network, and based on the modified attention parameters and other network parameters for embedding the input image, perform embedding processing (i.e., feature extraction processing) on ​​the second sample image (i.e., the input image of the attention mechanism network at this time), and generate a sample embedding feature of the second sample image. Optionally, the sample embedding feature can be a feature vector. The sample embedding feature is the feature learned by the attention mechanism network for the second sample image.

[0121] Optionally, there may be multiple second sample images. The multiple second sample images may be images obtained by photographing the sample palmprint from multiple shooting angles. One shooting angle is used to photograph a second sample image of the sample palmprint. Therefore, the process of generating the sample embedding feature in this application may include:

[0122] The image processing device can call the attention mechanism network to embed each second sample image based on the above-mentioned corrected attention parameters and other network parameters used to embed the input image, and generate an image embedding feature for each second sample image. A second sample image can have an image embedding feature, and the image embedding feature of a second sample image is that feature learned by the attention mechanism network for the second sample image.

[0123] Furthermore, the image processing device can fuse the image embedding features of the above-mentioned multiple second sample images to generate the above-mentioned sample embedding features. The sample embedding features can be the features obtained by fusing the features of the multi-angle photographed images of the sample palmprint (such as multiple second sample images).

[0124] It can be understood that since the second sample images obtained at various shooting angles can contain images of different clear areas of the sample palmprint, the sample embedding features obtained by fusing the second sample images obtained at various shooting angles can be made to contain the features of the images of the clear areas of the sample palmprint at various shooting angles. The features of the images of the clear areas are more accurate than the features of the images of the blurred areas. Therefore, the sample embedding features generated in this way will also be more accurate, and the sample embedding features can be used to subsequently achieve more accurate classification processing of the palmprint category of the second sample image.

[0125] Optionally, methods for fusing the image embedding features of the multiple second sample images include, but are not limited to, the following methods: splicing the image embedding features of the multiple second sample images (e.g., horizontal splicing) to generate a sample embedding feature; or summing the image embedding features of the multiple second sample images to generate a sample embedding feature. Optionally, the generated sample embedding feature may also be a feature vector.

[0126] Among them, the process of adding and processing the image embedding features of the multiple second sample images may include: the dimensions of the image embedding features of each second sample image may be the same, and therefore, the feature values ​​at the same position in the image embedding features of each second sample image may be added to generate a sample embedding feature.

[0127] Step S203: calling the classification sub-network to perform classification prediction processing on the sample palmprint based on the sample embedding features to obtain the classification prediction result of the sample palmprint.

[0128] Optionally, the palmprint classification network may further include a classification subnetwork, which may be used to classify the palmprints in the palmprint image to output a probability (which may constitute a probability distribution) that the palmprints in the palmprint image belong to each user (which may be referred to as an object) in a user set (which may be a set of all users to be identified in the palmprint recognition application scenario). Optionally, the classification subnetwork may be a fully connected network.

[0129] Therefore, the palmprint classification network can call the classification subnetwork to perform classification prediction processing on the sample palmprint through the above-mentioned sample embedding features, and obtain the classification prediction result of the sample palmprint. The classification prediction result can be a probability distribution (which can be expressed as a vector) composed of the probabilities that the predicted sample palmprint belongs to each user in the user set.

[0130] Step S204: Based on the difference between the true category and the classification prediction result, the network parameters in the palmprint classification network except for the frozen corrected attention parameters are corrected to obtain a trained palmprint classification network.

[0131] Optionally, the image processing device can correct other network parameters in the palmprint classification network except the frozen corrected attention parameters (which may include network parameters of the attention mechanism network for embedding the image except the corrected attention parameters, and network parameters of the classification subnetwork) by using the difference between the true category of the above-mentioned sample palmprint (which can also be expressed as a vector, in which the probability of the user indicated by the true category in the vector can be 1, and the probability of other users can be 0) and the classification prediction result (which can be reflected by cross entropy loss) to obtain a trained palmprint classification network.

[0132] If the palmprint classification network can be iteratively trained continuously through the above process until the network parameters of the palmprint classification network (other network parameters except the frozen modified attention parameters) reach a convergence state, or the number of iterative training of the palmprint classification network is greater than or equal to the number threshold, the palmprint classification network trained at this time can be regarded as the trained palmprint classification network.

[0133] Among them, the attention mechanism network in the trained palmprint classification network is the trained attention mechanism network, and the classification sub-network in the trained palmprint classification network is the trained classification sub-network.

[0134] Please refer to Figure 7, which is a schematic diagram of a scenario for training a palmprint classification network provided by an embodiment of the present application. As shown in Figure 7, first, the image processing device can correct the attention parameters of the attention mechanism network in the palmprint classification network using the first sample image.

[0135] Furthermore, the second sample image can be used to correct other network parameters in the palmprint classification network except the corrected attention parameters (including other feature embedding features of the attention mechanism network except the corrected attention parameters, and the network parameters of the classification sub-network for palmprint classification), and finally a trained palmprint classification network can be obtained.

[0136] Through the above process, a palmprint classification network can be trained. The trained palmprint classification network can include a trained attention mechanism network and a trained classification sub-network. Subsequently, the trained palmprint classification network can be used to accurately classify palmprint images, as described below.

[0137] The image processing device can obtain a palmprint image to be classified. The palmprint image can be an image obtained by photographing a target palmprint (which can be the palmprint of any user). Optionally, there can be multiple palmprint images to be classified. The multiple palmprint images can be images of the target palmprint photographed from multiple shooting angles. Each shooting angle can capture a palmprint image of the target palmprint.

[0138] The image processing device can call the trained attention mechanism network to perform embedding processing on the palmprint image to be classified to generate a target embedding feature of the palmprint image to be classified. The generation principle of the target embedding feature is the same as that of the sample embedding feature, which may include:

[0139] The image processing device can call the trained attention mechanism network to perform embedding processing on each palmprint image to be classified respectively to generate the image embedding features of each palmprint image. Then, the image embedding features of the multiple palmprint images to be classified can be fused (such as splicing or summing) to generate the above-mentioned target embedding features.

[0140] Among them, since the trained attention mechanism network includes the above-mentioned corrected attention parameters, the trained attention mechanism network can pay more attention to the images of clear areas in each palmprint image through the corrected attention parameters, thereby generating more accurate target embedding features for the palmprint image to be classified.

[0141] Please refer to Figure 8, which is a schematic diagram of a scenario for generating target embedding features provided by an embodiment of the present application. As shown in Figure 8, the palmprint images to be classified may include palmprint images 1 to m captured at multiple shooting angles. The image processing device can call a trained attention mechanism network to perform embedding processing on each palmprint image to be classified to generate image embedding feature 1 for palmprint image 1, image embedding feature 2 for palmprint image 2, image embedding feature 3 for palmprint image 3, ..., and image embedding feature m for palmprint image m.

[0142] Furthermore, feature fusion processing can be performed on the above-generated image embedding feature 1, image embedding feature 2, image embedding feature 3, ... and image embedding feature m to generate the target embedding feature.

[0143] Furthermore, the image processing device can call the trained classification subnetwork to perform classification and prediction processing on the target palmprint through the target embedding features generated above, and obtain the target category of the target palmprint. The target category can represent the user corresponding to the maximum probability among the probabilities predicted by the trained classification subnetwork that the target palmprint belongs to each user in the user set. This user is the owner of the predicted target palmprint, that is, the target category can be used to indicate the owner of the classified target palmprint.

[0144] Please refer to Figure 9, which is a schematic diagram of a palmprint prediction scenario provided by an embodiment of the present application. As shown in Figure 9, the image processing device can input the palmprint image to be classified into a trained palmprint classification network to call the trained attention mechanism network in the trained palmprint classification image to extract the target embedded features of the palmprint image to be classified.

[0145] Furthermore, the trained classification subnetwork in the trained palmprint classification image can be called to obtain the target category of the target palmprint in the palmprint image to be classified through classification prediction using the target embedding features generated above.

[0146] In a feasible implementation, the palm print image may be a palm print image captured in real time by a palm print acquisition device (such as a device that initiates the execution request for the target service) after a service initiator (which may be a service account, which may belong to a service user) initiates (such as initiating in a service platform) an execution request for the target service.

[0147] Therefore, if the target category indicates that the owner of the target palm print is the service initiator (ie, the service user to which the service initiator belongs), the target service can be executed (processed) for the service initiator, ie, the target service can be provided to the service initiator.

[0148] Through the above process, after achieving accurate classification and recognition of palm print images, corresponding security processing can be performed on related businesses.

[0149] Please refer to Figure 10, which is a schematic diagram of the structure of an image processing device provided by an embodiment of the present application. As shown in Figure 10, the image processing device 1 may include: a first acquisition module 11, a calling module 12, an adding module 13, a second acquisition module 14 and a correction module 15.

[0150] A first acquisition module 11 is configured to acquire a first sample image, wherein the first sample image includes a plurality of divided regional images. If the regional images have different sharpnesses, the regional images belong to different sharpness types.

[0151] a calling module 12, configured to call an attention mechanism network to perform attention degree recognition processing on the plurality of region images, to obtain a first region image and a second region image from the plurality of region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter;

[0152] an adding module 13, configured to add a first prediction label to the first region image and a second prediction label to the second region image based on the attention of the attention mechanism network to each of the first region image and the second region image, wherein the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, and the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type;

[0153] A second acquisition module 14 is configured to acquire a first reference label of the first region image and a second reference label of the second region image, wherein the first reference label is used to indicate that the first region image belongs to a first reference definition type, and the second reference label is used to indicate that the second region image belongs to a second reference definition type;

[0154] The correction module 15 is used to correct the attention parameter based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label. The attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

[0155] Optionally, the calling module 12 calls the attention mechanism network to perform attention recognition processing on the multiple region images to obtain the first region image and the second region image, including:

[0156] The attention mechanism network is called to determine the attention weight of each regional image in the multiple regional images based on the attention parameters. The attention weight of any regional image is used to reflect the attention degree of the attention mechanism network to the any regional image.

[0157] The region images having a focus weight within a first weight range among the plurality of region images are used as first region images, and the region images having a focus weight within a second weight range among the plurality of region images are used as second region images;

[0158] The weight value in the first weight range is greater than the weight value in the second weight range.

[0159] Optionally, the attention mechanism network is included in the palmprint classification network, the modified attention parameters are frozen in the palmprint classification network, the palmprint classification network further includes a classification subnetwork, and the above-mentioned device 1 is further used for:

[0160] Acquire a second sample image, where the second sample image is an image captured by capturing the sample palmprint, and the second sample image has a palmprint label, where the palmprint label is used to indicate a true category of the sample palmprint;

[0161] Calling the attention mechanism network to embed the second sample image based on the corrected attention parameters to generate a sample embedding feature of the second sample image;

[0162] The classification sub-network is called to perform classification prediction on the sample palmprint based on the sample embedding features to obtain the classification prediction result of the sample palmprint;

[0163] Based on the difference between the true category and the classification prediction result, the network parameters in the palmprint classification network except the frozen corrected attention parameters are corrected to obtain a trained palmprint classification network.

[0164] Optionally, there are multiple second sample images, and the multiple second sample images are images obtained by photographing the sample palmprint from multiple shooting angles, and one shooting angle is used to photograph and obtain one second sample image of the sample palmprint;

[0165] The method in which the apparatus 1 calls the attention mechanism network to perform embedding processing on the second sample image based on the modified attention parameters to generate a sample embedding feature of the second sample image includes:

[0166] Calling the attention mechanism network to embed each second sample image based on the corrected attention parameters to generate image embedding features for each second sample image;

[0167] The image embedding features of the plurality of second sample images are fused to generate a sample embedding feature.

[0168] Optionally, the apparatus 1 performs fusion processing on the image embedding features of the plurality of second sample images to generate a sample embedding feature, including:

[0169] performing splicing processing on the image embedding features of the plurality of second sample images to generate a sample embedding feature; or,

[0170] The image embedding features of the plurality of second sample images are summed to generate a sample embedding feature.

[0171] Optionally, the trained palmprint classification network includes a trained attention mechanism network and a trained classification sub-network, and the apparatus 1 is further used for:

[0172] Obtaining a palmprint image to be classified, where the palmprint image is an image obtained by photographing the target palmprint;

[0173] Call the trained attention mechanism network to embed the palmprint image and generate the target embedding features of the palmprint image;

[0174] The trained classification subnetwork is called to perform classification prediction processing on the target palmprint based on the target embedded features to obtain the target category of the target palmprint.

[0175] Optionally, there are multiple palmprint images, and the multiple palmprint images are images obtained by photographing the target palmprint from multiple shooting angles, and one shooting angle is used to photograph and obtain one palmprint image of the target palmprint;

[0176] The above-mentioned device 1 calls the trained attention mechanism network to embed the palmprint image and generate the target embedding features of the palmprint image, including:

[0177] Call the trained attention mechanism network to embed each palmprint image separately and generate the image embedding features of each palmprint image;

[0178] The image embedding features of multiple palmprint images are fused to generate target embedding features.

[0179] Optionally, the target category is used to indicate the owner of the target palmprint. The palmprint image is captured after the service initiator initiates an execution request for the target service.

[0180] The above device 1 is also used for:

[0181] If the target category is used to indicate that the owner of the target palmprint is a service initiator, the target service is executed on the service initiator.

[0182] Optionally, the second acquisition module 14 acquires the first reference label of the first region image and the second reference label of the second region image, including:

[0183] Calling a trained definition classification network to perform classification and prediction processing on the definition of the first region image to obtain a first reference definition type of the first region image;

[0184] Calling the trained definition classification network to perform classification and prediction processing on the definition of the second region image to obtain a second reference definition type of the second region image;

[0185] A first reference label is added to the first region image based on the first reference definition type, and a second reference label is added to the second region image based on the second reference definition type.

[0186] Optionally, the device 1 is further used for:

[0187] Obtaining a third sample image and a definition classification network to be trained, wherein the third sample image has a definition label, and the definition label is used to indicate a true definition type of the third sample image;

[0188] Calling the definition classification network to be trained to perform classification and prediction processing on the definition type of the third sample image to obtain a predicted definition type of the third sample image;

[0189] Based on the difference between the actual clarity type and the predicted clarity type, the network parameters of the clarity classification network to be trained are modified to obtain a trained clarity classification network.

[0190] Optionally, the second acquisition module 14 acquires the first reference label of the first region image and the second reference label of the second region image, including:

[0191] Calling the trained image segmentation network to perform image segmentation processing on the first sample image to obtain multiple segmented images of the first sample image, where each segmented image corresponds to a definition type;

[0192] Using the segmented image with the highest overlap with the first region image among the multiple segmented images as the first definition matching image of the first region image, and adding a first reference label to the first region image based on the definition type corresponding to the first definition matching image; and

[0193] using a segmented image having the highest overlap with the second region image among the multiple segmented images as a second definition matching image of the second region image, and adding a second reference label to the second region image based on the definition type corresponding to the second definition matching image;

[0194] The first reference definition type is the definition type corresponding to the first definition matching image, and the second reference definition type is the definition type corresponding to the second definition matching image.

[0195] According to one embodiment of the present application, the steps involved in the image processing method shown in FIG3 can be performed by various modules in the image processing device 1 shown in FIG10. For example, step S101 shown in FIG3 can be performed by the first acquisition module 11 in FIG10, and step S102 shown in FIG3 can be performed by the calling module 12 in FIG10; step S103 shown in FIG3 can be performed by the adding module 13 in FIG10, and step S104 shown in FIG3 can be performed by the second acquisition module 14 in FIG10; and step S105 shown in FIG3 can be performed by the correcting module 15 in FIG10.

[0196] The present application can obtain a first sample image; the first sample image includes a plurality of divided regional images, and if the clarity of the regional images is different, the clarity types of the regional images are different; and the attention mechanism network can be called to perform attention recognition processing on the plurality of regional images to obtain a first regional image and a second regional image in the plurality of regional images; the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention to the first regional image is higher than the attention to the second regional image based on the attention parameter; furthermore, based on the attention of the attention mechanism network to the respective attention of the first regional image and the second regional image, a first prediction label can be added to the first regional image, and a second prediction label can be added to the second regional image; the first prediction label is used to indicate that the first regional image belongs to the first predicted clarity type; the second prediction label is used to indicate that the second area image belongs to the second prediction clarity type; the clarity indicated by the first prediction clarity type is higher than the clarity indicated by the second prediction clarity type; a first reference label of the first area image and a second reference label of the second area image can also be obtained; the first reference label is used to indicate that the first area image belongs to the first reference clarity type; the second reference label is used to indicate that the second area image belongs to the second reference clarity type; thereby, the attention parameter can be corrected based on the difference between the first prediction label and the first reference label, and the difference between the second prediction label and the second reference label; the attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image. It can be seen that the device proposed in this application can add prediction labels to the regional images of the first sample image by paying attention to the regional images in the first sample image through the attention mechanism network, and the clarity of the clarity type indicated by the prediction label added to the regional image with higher attention can be higher. Then, the attention parameter can be corrected by the difference between the real label of the regional image (such as the reference label) and the predicted label. Subsequently, the attention mechanism network can use the corrected attention parameter to pay higher attention to the regional images with higher clarity in the palmprint image, and pay lower attention to the regional images with lower clarity in the palmprint image. In this way, the image features of the regional images with higher clarity in the palmprint image can be extracted to the greatest extent, and finally more accurate embedding features of the palmprint image can be extracted. Through the more accurate embedding features of the palmprint image, more accurate classification processing of the palmprint image can be achieved (that is, more accurate identification processing of the owner of the palmprint in the palmprint image).

[0197] According to one embodiment of the present application, the various modules in the image processing device 1 shown in Figure 10 can be individually or all combined into one or several units to constitute, or one (some) of the units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In actual applications, the functions of a module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the image processing device 1 may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0198] According to one embodiment of the present application, a computer program capable of executing the steps of the corresponding methods described in the various embodiments of the present application can be run on a general-purpose computer device (the computer device may include processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM)) to construct the image processing device 1 shown in FIG10 . The computer program can be recorded on a computer-readable recording medium, for example, and can be loaded into the computer device via the computer-readable recording medium and run therein.

[0199] Please refer to Figure 11, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 11, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may further include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 11, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0200] In the computer device 1000 shown in FIG11 , the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0201] Acquire a first sample image, wherein the first sample image includes a plurality of divided regional images, and if the regional images have different sharpnesses, the regional images belong to different sharpness types;

[0202] Invoking an attention mechanism network to perform attention degree recognition processing on the multiple region images to obtain a first region image and a second region image from the multiple region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter;

[0203] Based on the attention mechanism network's respective attention levels to the first region image and the second region image, a first prediction label is added to the first region image, and a second prediction label is added to the second region image, where the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, where the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type;

[0204] Acquire a first reference tag of the first region image and a second reference tag of the second region image, wherein the first reference tag is used to indicate that the first region image belongs to a first reference definition type, and the second reference tag is used to indicate that the second region image belongs to a second reference definition type;

[0205] Based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label, the attention parameter is modified, and the attention mechanism network is used to extract the embedded features of the palmprint image based on the modified attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

[0206] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the image processing methods described in the various embodiments of the present application, and can also execute the image processing apparatus 1 described in the embodiment corresponding to FIG. 10 , and thus will not be repeated here. Furthermore, the beneficial effects of employing the same methods will not be repeated here.

[0207] In addition, it should be noted that this application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When a processor executes the computer program, it can perform the description of the image processing method in each embodiment of this application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0208] As an example, the above-mentioned computer program can be deployed and executed on one computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network. Multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain network.

[0209] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped with the computer device. Furthermore, the computer-readable storage medium may include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is about to be output.

[0210] The present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the description of the above-mentioned image processing method in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application.

[0211] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0212] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0213] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first sample image, wherein the first sample image includes a plurality of divided regional images, and if the regional images have different sharpness, the regional images belong to different sharpness types; Calling an attention mechanism network to perform attention degree recognition processing on the multiple region images to obtain a first region image and a second region image among the multiple region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter; Based on the attention of the attention mechanism network to the first region image and the second region image, a first prediction label is added to the first region image, and a second prediction label is added to the second region image, wherein the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, and the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type; Acquire a first reference tag of the first region image and a second reference tag of the second region image, wherein the first reference tag is used to indicate that the first region image belongs to a first reference definition type, and the second reference tag is used to indicate that the second region image belongs to a second reference definition type; Based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label, the attention parameter is corrected, and the attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

2. The method according to claim 1, characterized in that The calling of the attention mechanism network to perform attention recognition processing on the multiple region images to obtain a first region image and a second region image includes: Calling the attention mechanism network to determine an attention weight for each of the multiple regional images based on the attention parameter, wherein the attention weight of any regional image is used to reflect the attention degree of the attention mechanism network to the any regional image; Using a region image whose attention weight is within a first weight range among the plurality of region images as the first region image, and using a region image whose attention weight is within a second weight range among the plurality of region images as the second region image; Among them, the weight value in the first weight range is greater than the weight value in the second weight range.

3. The method according to claim 1 or 2, characterized in that The attention mechanism network is included in the palmprint classification network, the modified attention parameter is frozen in the palmprint classification network, the palmprint classification network further includes a classification subnetwork, and the method further includes: Acquire a second sample image, where the second sample image is an image obtained by photographing a sample palmprint, and the second sample image has a palmprint label, where the palmprint label is used to indicate a true category of the sample palmprint; Calling the attention mechanism network to perform embedding processing on the second sample image based on the corrected attention parameter to generate a sample embedding feature of the second sample image; Calling the classification subnetwork to perform classification prediction processing on the sample palmprint based on the sample embedding feature to obtain a classification prediction result of the sample palmprint; Based on the difference between the true category and the classification prediction result, the network parameters in the palmprint classification network except the frozen modified attention parameters are modified to obtain a trained palmprint classification network.

4. The method according to any one of claims 1 to 3, characterized in that: There are multiple second sample images, and the multiple second sample images are images obtained by photographing the sample palm print from multiple shooting angles, and one shooting angle is used to photograph a second sample image of the sample palm print; The calling of the attention mechanism network to perform embedding processing on the second sample image based on the modified attention parameter to generate a sample embedding feature of the second sample image includes: Call the attention mechanism network to embed each second sample image based on the modified attention parameters, and generate forming an image embedding feature of each second sample image; The image embedding features of the plurality of second sample images are fused to generate the sample embedding features.

5. The method according to any one of claims 1 to 4, characterized in that: The fusing the image embedding features of the plurality of second sample images to generate the sample embedding features comprises: performing splicing processing on the image embedding features of the plurality of second sample images to generate the sample embedding features; or, The image embedding features of the plurality of second sample images are summed up to generate the sample embedding features.

6. The method according to any one of claims 1 to 5, characterized in that: The trained palmprint classification network includes a trained attention mechanism network and a trained classification sub-network, and the method further includes: Acquire a palmprint image to be classified, wherein the palmprint image is an image obtained by photographing a target palmprint; Calling the trained attention mechanism network to embed the palmprint image and generate target embedding features of the palmprint image; The trained classification subnetwork is called to perform classification prediction processing on the target palmprint based on the target embedding feature to obtain the target category of the target palmprint.

7. The method according to any one of claims 1 to 6, characterized in that: There are multiple palm print images, which are images obtained by photographing the target palm print from multiple shooting angles, and one shooting angle is used to photograph a palm print image of the target palm print; The calling of the trained attention mechanism network to perform embedding processing on the palmprint image to generate a target embedding feature of the palmprint image includes: Calling the trained attention mechanism network to perform embedding processing on each palmprint image respectively, and generating image embedding features of each palmprint image; The image embedding features of the multiple palmprint images are fused to generate the target embedding feature.

8. The method according to any one of claims 1 to 7, characterized in that: The target category is used to indicate the owner of the target palm print, and the palm print image is taken after the service initiator initiates an execution request for the target service; The method further comprises: If the target category is used to indicate that the owner of the target palm print is the service initiator, the target service is executed on the service initiator.

9. The method according to any one of claims 1 to 8, characterized in that The acquiring a first reference label of the first area image and a second reference label of the second area image comprises: Calling a trained definition classification network to perform classification prediction processing on the definition of the first region image to obtain the first reference definition type of the first region image; Calling the trained definition classification network to perform classification prediction processing on the definition of the second region image to obtain the second reference definition type of the second region image; The first reference tag is added to the first area image based on the first reference definition type, and the second reference tag is added to the second area image based on the second reference definition type.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: Acquire a third sample image and a definition classification network to be trained, wherein the third sample image has a definition label, and the definition label is used to indicate a true definition type of the third sample image; Calling the definition classification network to be trained to perform classification prediction processing on the definition type of the third sample image to obtain a predicted definition type of the third sample image; Based on the difference between the actual definition type and the predicted definition type, the network parameters of the definition classification network to be trained are modified to obtain the trained definition classification network.

11. The method according to any one of claims 1 to 10, characterized in that: The acquiring a first reference label of the first area image and a second reference label of the second area image comprises: Calling the trained image segmentation network to perform image segmentation processing on the first sample image to obtain multiple segmented images of the first sample image, where one segmented image corresponds to one definition type; Using the segmented image with the highest overlap with the first region image among the multiple segmented images as the first definition matching image of the first region image, and adding the first reference label to the first region image based on the definition type corresponding to the first definition matching image; and using a segmented image having the highest overlap with the second region image among the multiple segmented images as a second definition matching image of the second region image, and adding the second reference label to the second region image based on a definition type corresponding to the second definition matching image; The first reference definition type is a definition type corresponding to the first definition matching image, and the second reference definition type is a definition type corresponding to the second definition matching image.

12. An image processing device, characterized in that: The device comprises: A first acquisition module is used to acquire a first sample image, wherein the first sample image includes a plurality of divided regional images, and if the regional images have different sharpness, the regional images belong to different sharpness types; A calling module, configured to call an attention mechanism network to perform attention degree recognition processing on the multiple region images to obtain a first region image and a second region image from the multiple region images, wherein the attention mechanism network includes an attention parameter, and the attention mechanism network determines that the attention degree of the first region image is higher than the attention degree of the second region image based on the attention parameter; an adding module, configured to add a first prediction label to the first region image and a second prediction label to the second region image based on the attention degree of the attention mechanism network to the first region image and the second region image, wherein the first prediction label is used to indicate that the first region image belongs to a first prediction definition type, and the second prediction label is used to indicate that the second region image belongs to a second prediction definition type, and the definition indicated by the first prediction definition type is higher than the definition indicated by the second prediction definition type; a second acquisition module, configured to acquire a first reference tag of the first region image and a second reference tag of the second region image, wherein the first reference tag is used to indicate that the first region image belongs to a first reference definition type, and the second reference tag is used to indicate that the second region image belongs to a second reference definition type; A correction module is used to correct the attention parameter based on the difference between the first predicted label and the first reference label, and the difference between the second predicted label and the second reference label. The attention mechanism network is used to extract the embedded features of the palmprint image based on the corrected attention parameters, and the embedded features are used to identify the owner of the palmprint in the palmprint image.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 11 when being executed by a processor.

14. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 11.

Images