Image processing method and device, terminal equipment and computer readable storage medium

Abstract

The application is suitable for the technical field of image processing, and provides an image processing method and device, terminal equipment and computer readable storage medium, comprising: extracting feature information of each feature channel in a to-be-processed image to obtain a feature map, wherein one feature channel corresponds to one image feature; dividing feature information of feature channels belonging to the same image feature in the feature map into one feature group; extracting feature information corresponding to each feature group to obtain local features; and generating global features of the to-be-processed image according to the local features corresponding to each feature group, wherein the global features are used for identifying target objects in the to-be-processed image. Through the method in the embodiment of the application, the accuracy of the image recognition result can be improved.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image processing method, apparatus, terminal device and computer-readable storage medium. Background Technology

[0002] Image recognition refers to the technology of using computers to process, analyze, and understand images in order to identify targets and objects of various patterns. Image recognition technology is widely used in various fields, such as face recognition, vehicle recognition, license plate recognition, and pedestrian re-identification. Image recognition tasks rely on high-quality feature information of the target object. Therefore, how to extract high-quality features of the target object has always been a challenge in image recognition technology.

[0003] Existing feature extraction methods are unable to specifically extract feature information of target objects. In particular, when target object detection is inaccurate due to changes in external lighting or target object pose, the feature information extracted by existing methods is of low quality, resulting in low accuracy of image recognition results. Summary of the Invention

[0004] This application provides an image processing method, apparatus, terminal device, and computer-readable storage medium, which can improve the accuracy of image recognition results.

[0005] In a first aspect, embodiments of this application provide an image processing method, including:

[0006] Extract the feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein each feature channel corresponds to an image feature;

[0007] The feature information of feature channels belonging to the same image features in the feature map is divided into a feature group;

[0008] Extract the feature information corresponding to each feature group to obtain local features;

[0009] The global features of the image to be processed are generated based on the local features corresponding to each feature group, and the global features are used to identify the target object in the image to be processed.

[0010] In this embodiment of the application, the feature information of the feature channels belonging to the same image features in the feature map is divided into a feature group, which is equivalent to clustering the feature channels of the feature map. The feature information in the obtained feature group includes the complete structural information of the target object, avoiding the destruction of the consistency of the target object structure, thereby helping to improve the accuracy of target recognition.

[0011] In one possible implementation of the first aspect, the size of the feature map is W×H×N, where W represents the width of the feature map, H represents the height of the feature map, and N represents the number of feature channels corresponding to the feature map;

[0012] The step of dividing the feature information of feature channels belonging to the same image features in the feature map into a feature group includes:

[0013] The two-dimensional plane of the feature map is divided into at least two planar regions, wherein the two-dimensional plane is represented by W×H, and the number of planar regions is equal to the number of feature groups;

[0014] Detect the planar region to which the feature information of each feature channel in the feature map belongs;

[0015] The feature information of feature channels belonging to the same planar region is divided into a feature group.

[0016] In this embodiment, the feature channels are grouped according to the structural features of the target object. This grouping method can better highlight the features of each structure of the target object when extracting local features in the future, which is conducive to improving the target recognition accuracy.

[0017] In one possible implementation of the first aspect, dividing the two-dimensional plane of the feature map into at least two planar regions includes:

[0018] Based on the structural features of the target object in the image to be processed, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0019] In this embodiment, the two-dimensional plane of the feature map is divided into regions according to the structural features of the target object, so that the target object can still be accurately located even if the target object detection is inaccurate.

[0020] In one possible implementation of the first aspect, dividing the two-dimensional plane of the feature map into at least two planar regions based on the structural features of the target object in the image to be processed includes:

[0021] Identify the structural components of the target object in the image to be processed, and obtain a detection box corresponding to each structural component;

[0022] Based on the position of the detection box corresponding to each of the structural components, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0023] In one possible implementation of the first aspect, detecting the planar region to which the feature information of each feature channel in the feature map belongs includes:

[0024] Obtain the target value from the feature information of each feature channel in the feature map;

[0025] Based on the planar region to which the target value belongs, determine the planar region to which the feature information of each feature channel of the feature map belongs.

[0026] Since different feature channels focus on extracting feature information of different image types from the image to be processed, the distribution of feature values ​​in the feature information of different feature channels is also different. Using the target value in the feature information to determine the corresponding planar region fully reflects the distribution law of the features.

[0027] In one possible implementation of the first aspect, generating global features of the image to be processed based on local features corresponding to each of the feature groups includes:

[0028] The local features corresponding to each feature group are concatenated to generate a vector, which is the global feature.

[0029] In one possible implementation of the first aspect, the method further includes:

[0030] The target object in the image to be processed is identified based on the global features, and the identification result is obtained.

[0031] Secondly, embodiments of this application provide an image processing apparatus, including:

[0032] A global extraction unit is used to extract feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein one feature channel corresponds to one image feature;

[0033] The feature segmentation unit is used to divide the feature information of feature channels belonging to the same image feature in the feature map into a feature group;

[0034] A local extraction unit is used to extract feature information corresponding to each feature group to obtain local features;

[0035] The feature generation unit is used to generate global features of the image to be processed based on the local features corresponding to each feature group, and the global features are used to identify target objects in the image to be processed.

[0036] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method as described in any one of the first aspects above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image processing method as described in any one of the first aspects above.

[0038] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the image processing method described in any one of the first aspects.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the pedestrian feature extraction algorithm based on the average division method provided in the embodiments of this application;

[0042] Figure 2 This is a schematic diagram of a pedestrian detection image provided in an embodiment of this application;

[0043] Figure 3 This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0044] Figure 4 This is a schematic diagram of the planar region division of the image to be processed provided in an embodiment of this application;

[0045] Figure 5 This is a schematic flowchart of an image processing method provided in another embodiment of this application;

[0046] Figure 6 This is a structural block diagram of the image processing apparatus provided in the embodiments of this application;

[0047] Figure 7 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0049] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0050] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0051] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0052] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0054] This application first introduces the technical background of its embodiments. Image recognition refers to the technology of using computers to process, analyze, and understand images to identify targets and objects of various patterns. Image recognition technology is widely used in various fields, such as face recognition, vehicle recognition, license plate recognition, and pedestrian re-identification. Image recognition tasks rely on high-quality feature information of target objects. Therefore, how to extract high-quality features of target objects has always been a challenge in image recognition technology. Existing feature extraction methods cannot specifically extract feature information of target objects. Especially when target object detection is inaccurate due to changes in external lighting or changes in the pose of the target object, the feature information of the target object extracted by existing methods is of low quality, resulting in low accuracy of image recognition results.

[0055] Taking pedestrian re-identification as an example, the task is to determine whether a specific pedestrian exists in an image or video. Existing technologies typically use an average division method to extract pedestrian features. See also... Figure 1 This is a flowchart illustrating the pedestrian feature extraction algorithm based on average partitioning provided in an embodiment of this application. Figure 1 As shown, the pedestrian feature extraction algorithm based on the average division method mainly consists of three steps: global feature map generation, component feature extraction, and global feature synthesis.

[0056] The first step is global feature map generation: For an input image, a deep neural network is used to generate its global feature map. The deep network used here can be any common deep learning network, such as heavyweight networks like ResNet and lightweight networks like MobileNet and ShuffleNet.

[0057] The second step is component feature extraction: the global feature map is divided into 6 local regions along the vertical direction, and the corresponding local features are extracted from these 6 local regions.

[0058] The third step is global feature synthesis: generating global features of pedestrians from 6 sets of local features.

[0059] Because the above methods involve averaging the global feature map, when pedestrian detection is inaccurate, this averaging approach often fails to precisely locate human body components (i.e., body parts). See also Figure 2 This is a schematic diagram of a pedestrian detection image provided in an embodiment of this application. Figure 2As shown, the left image is the detection image obtained when pedestrian detection is accurate, and the right image is the detection image obtained when pedestrian detection is inaccurate. The comparison reveals that if the right image is divided into equal vertical regions, the top two regions contain no human components. In this case, background features, not pedestrian features, are extracted from the top two regions of the right image. This results in background features being mixed into the final generated global features, reducing the quality of the global features and consequently lowering the accuracy of pedestrian recognition.

[0060] To address the aforementioned problems, embodiments of this application provide an image processing method. See also... Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0061] S301, extract the feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein each feature channel corresponds to an image feature.

[0062] In this embodiment, the feature map has a size of W×H×N, where W represents the width of the feature map, H represents the height of the feature map, and N represents the number of feature channels corresponding to the feature map. In some examples, the feature map can be represented as a three-dimensional matrix, where the elements of the matrix represent feature values.

[0063] The feature information of all feature channels can represent all the features of the image to be processed. Therefore, the feature map here can be regarded as the global features of the image to be processed.

[0064] For example, taking color channels as an example, an RGB image can be divided into three channels: red, green, and blue. Accordingly, extracting the feature information of each of the three color channels of the image yields a feature map with dimensions W×H×3, where 3 represents the three feature channels.

[0065] For example, in human body recognition applications, the human body structure can be divided into the head, upper body, and lower body. Each structure can be further divided into three colors: red, green, and blue. Each structure and its corresponding three colors can be used as a feature channel. This results in nine feature channels, representing the following features: head-red, head-green, head-blue, upper body-red, upper body-green, upper body-blue, lower body-red, lower body-green, and lower body-blue. The resulting feature map has dimensions of W×H×9, where 9 represents the nine feature channels.

[0066] It should be noted that the above are merely examples of feature channels. In practical applications, the number of feature channels can be set as needed. Commonly used number of feature channels include 1024 and 2048. In this embodiment, no specific limitations are made on the number of feature channels or the image features represented by the feature channels.

[0067] S302, the feature information of the feature channels belonging to the same image features in the feature map is divided into a feature group.

[0068] Continuing with the aforementioned human body recognition application scenario, which includes nine feature channels, the feature information of feature channels belonging to the same human body structural component can be grouped into a feature group. Specifically, the feature information of the head-red, head-green, and head-blue feature channels can be grouped into one feature group; the feature information of the upper body-red, upper body-green, and upper body-blue feature channels can be grouped into another feature group; and the feature information of the lower body-red, lower body-green, and lower body-blue feature channels can be grouped into another feature group. Alternatively, the feature information of feature channels belonging to the same color can also be grouped into one feature group. The division of feature groups can be determined based on the characteristics of the target object to be identified.

[0069] In some embodiments, S302 may include:

[0070] I. Divide the two-dimensional plane of the feature map into at least two planar regions, wherein the two-dimensional plane is represented by W×H, and the number of planar regions is equal to the number of feature groups.

[0071] In this embodiment, the size of the two-dimensional plane of the feature map is the same as the size of the image to be processed. Figure 2 Each element in the 3D plane represents the feature value of the pixel in the image to be processed that corresponds to that element. In other words, for features... Figure 2 The division of a 2D plane is equivalent to the division of a planar region of the image to be processed.

[0072] In one implementation, step I includes:

[0073] The two-dimensional plane of the feature map is divided into at least two planar regions on average.

[0074] For example, taking human body recognition as an example, see Figure 4 This is a schematic diagram of the planar region division of the image to be processed provided in an embodiment of this application. For example... Figure 4 As shown in (a), the image to be processed is divided into five planar regions f1-f5 along the vertical direction. It is evident that when the target human detection is inaccurate, average division often fails to accurately locate human components. Figure 4 As shown in (a), the planar regions f1 and f2 do not contain any human body components. In this case, the local features of the feature groups corresponding to f1 and f2 extracted subsequently cannot provide valuable feature information for human body recognition.

[0075] To solve the above problem, in another implementation, step I includes:

[0076] Based on the structural features of the target object in the image to be processed, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0077] Optionally, the structural components of the target object in the image to be processed are identified, and a detection box corresponding to each structural component is obtained; based on the position of the detection box corresponding to each structural component, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0078] Specifically, if the detection boxes are distributed along the vertical direction of the image to be processed, then the straight line where the lower edge of each detection box is located is used as a dividing line.

[0079] like Figure 4 As shown in (b), the image to be processed is divided into three planar regions f1'-f3'. Figure 4 Compared to (a) in the middle, Figure 4 In (b), f1' is equivalent to dividing f1-f3 into a planar region. In this way, it is ensured that each planar region contains the structural features of the target object, avoiding interference from invalid feature information to subsequent target recognition, and thus improving the accuracy of target recognition.

[0080] In this embodiment, the two-dimensional plane of the feature map is divided into regions according to the structural features of the target object, so that the target object can still be accurately located even if the target object detection is inaccurate.

[0081] II. Detect the planar region to which the feature information of each feature channel in the feature map belongs.

[0082] Since different feature channels focus on extracting feature information of different image types in the image to be processed, the feature value distribution in the feature information of different feature channels is also different.

[0083] In one implementation, step II includes:

[0084] Obtain the target value from the feature information of each feature channel in the feature map;

[0085] Based on the planar region to which the target value belongs, determine the planar region to which the feature information of each feature channel of the feature map belongs.

[0086] Optionally, the target value can be the largest feature value or multiple large feature values.

[0087] For example, when the target value is the largest eigenvalue, determine the planar region to which the target value belongs. In one case, if the target value is located at the boundary of two planar regions, obtain the second largest eigenvalue, determine the region based on the location of the second largest eigenvalue, and so on. Alternatively, specify the region, such as the lower planar region among the two planar regions, as the region to which the target value belongs.

[0088] For example, when the target value is multiple large feature values, the position of each target value on the two-dimensional plane is determined, and then a feature region is determined based on the position of each target value on the two-dimensional plane. Finally, it is determined which planar region the feature region belongs to.

[0089] In one scenario, if the feature region is located within a planar region A, then the target value is determined to belong to that planar region A.

[0090] In another scenario, if the feature region is distributed across two planar regions A and B, the target value can be determined based on the area it occupies in each region. For example, if the area of ​​the feature region in planar region A is greater than its area in planar region B, then the target value is determined to belong to planar region A. Alternatively, the median of several larger feature values ​​can be calculated, and the planar region to which that median belongs can be identified as the target value.

[0091] III. Divide the feature information of feature channels belonging to the same planar region into a feature group.

[0092] In the embodiments described in steps I-III above, it is equivalent to grouping the feature channels according to the structural features of the target object. This grouping method can better highlight the features of each structure of the target object when extracting local features in the subsequent process, which is conducive to improving the target recognition accuracy.

[0093] S303, extract the feature information corresponding to each feature group to obtain local features.

[0094] In this embodiment, a trained feature extraction model can be used to extract local feature information for each first sub-image. The feature extraction model can be a neural network model or an algorithm model with feature extraction capabilities, etc.

[0095] S304, Generate global features of the image to be processed based on the local features corresponding to each feature group, wherein the global features are used to identify target objects in the image to be processed.

[0096] In one implementation, step S304 includes: concatenating the local features corresponding to each feature group to generate a vector, where the vector represents the global feature. The dimension of this vector is the sum of the dimensions of all local features.

[0097] Of course, all the local features can also be grouped into a matrix, where each column / row represents a set of local features.

[0098] In another implementation, S304 includes: weighted summation of each group of local features to generate global features.

[0099] The weights for each set of local features can be preset according to actual needs. For example, in a football match monitoring application, where the focus is on the legs, the weights for local features representing the legs can be set higher, while the weights for local features representing other parts of the body can be set lower. Similarly, in a traffic restriction monitoring application, where the focus is on license plate information, the weights for local features representing the front and rear of the vehicle can be set higher, while the weights for local features representing the passenger compartment can be set lower.

[0100] For example, see human body recognition. Figure 5 This is a schematic flowchart of an image processing method provided in another embodiment of this application. Figure 5 As shown, feature information from multiple feature channels is extracted from the image to be processed to obtain a feature map F'. The feature information of feature channels belonging to the same human body component in the feature map is divided into a feature group, resulting in 6 feature groups G1-G6. Local features f1'-f6' of each feature group are extracted respectively, and finally the local features are synthesized into a global feature f'.

[0101] Figure 1 Dividing the two-dimensional plane corresponding to each feature channel in the feature map into six regions is equivalent to cutting along the height direction of the feature map. This method is like segmenting the human body structure, disrupting its consistency. Figure 1 compared to, Figure 5 The method shown essentially groups multiple feature channels, and the feature information corresponding to each feature channel contains the complete human body structure, thus preserving the consistency of the human body structure.

[0102] In this embodiment of the application, the feature information of the feature channels belonging to the same image features in the feature map is divided into a feature group, which is equivalent to clustering the feature channels of the feature map. The feature information in the obtained feature group includes the complete structural information of the target object, avoiding the destruction of the consistency of the target object structure, thereby helping to improve the accuracy of target recognition.

[0103] In some embodiments, the method further includes:

[0104] The target object in the image to be processed is identified based on the global features, and the identification result is obtained.

[0105] One way to implement this step is to input global features into the trained recognition model and output the recognition result.

[0106] For example, in traffic violation monitoring applications, the identification result can be either a violation or no violation, and can also include the type of violation. In pedestrian re-identification applications, the identification result can indicate whether the person is a specific tracking target.

[0107] In this embodiment, each set of local features represents an image feature of the target object, while the global features include all image features of the target object. Using global features to identify the target object in the image to be processed is equivalent to considering the correlation between various image features of the target object, which helps improve the accuracy of target recognition.

[0108] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0109] Corresponding to the image processing method described in the above embodiments, Figure 6 This is a structural block diagram of the image processing apparatus provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0110] Reference Figure 6 The device includes:

[0111] The global extraction unit 61 is used to extract the feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein one feature channel corresponds to one image feature.

[0112] The feature segmentation unit 62 is used to divide the feature information of the feature channels belonging to the same image features in the feature map into a feature group.

[0113] The local extraction unit 63 is used to extract the feature information corresponding to each feature group to obtain local features.

[0114] The feature generation unit 64 is used to generate global features of the image to be processed based on the local features corresponding to each feature group, and the global features are used to identify target objects in the image to be processed.

[0115] Optionally, the size of the feature map is W×H×N, where W represents the width of the feature map, H represents the height of the feature map, and N represents the number of feature channels corresponding to the feature map.

[0116] Correspondingly, feature segmentation unit 62 is also used for:

[0117] The two-dimensional plane of the feature map is divided into at least two planar regions, wherein the two-dimensional plane is represented by W×H, and the number of planar regions is equal to the number of feature groups;

[0118] Detect the planar region to which the feature information of each feature channel in the feature map belongs;

[0119] The feature information of feature channels belonging to the same planar region is divided into a feature group.

[0120] Optionally, feature segmentation unit 62 is also used for:

[0121] Based on the structural features of the target object in the image to be processed, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0122] Optionally, feature segmentation unit 62 is also used for:

[0123] Identify the structural components of the target object in the image to be processed, and obtain a detection box corresponding to each structural component;

[0124] Based on the position of the detection box corresponding to each of the structural components, the two-dimensional plane of the feature map is divided into at least two planar regions.

[0125] Optionally, feature segmentation unit 62 is also used for:

[0126] Obtain the target value from the feature information of each feature channel in the feature map;

[0127] Based on the planar region to which the target value belongs, determine the planar region to which the feature information of each feature channel of the feature map belongs.

[0128] Optionally, the feature generation unit 64 is also used for:

[0129] The local features corresponding to each feature group are concatenated to generate a vector, which is the global feature.

[0130] Optionally, device 6 also includes:

[0131] The target recognition unit 65 is used to identify the target object in the image to be processed based on the global features and obtain the recognition result.

[0132] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0133] in addition, Figure 6 The image processing device shown can be a software unit, a hardware unit, or a combination of software and hardware built into an existing terminal device, or it can be integrated into the terminal device as an independent component, or it can exist as an independent terminal device.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] Figure 7 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 7 As shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described image processing method embodiments.

[0136] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 7 This is merely an example of terminal device 7 and does not constitute a limitation on terminal device 7. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0137] The processor 70 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0138] In some embodiments, the memory 71 may be an internal storage unit of the terminal device 7, such as a hard disk or memory of the terminal device 7. In other embodiments, the memory 71 may be an external storage device of the terminal device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 7. Furthermore, the memory 71 may include both internal and external storage units of the terminal device 7. The memory 71 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0139] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0140] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: Extract the feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein each feature channel corresponds to an image feature; The feature information of feature channels belonging to the same image features in the feature map is divided into a feature group; Extract the feature information corresponding to each feature group to obtain local features; A global feature of the image to be processed is generated based on the local features corresponding to each feature group, and the global feature is used to identify the target object in the image to be processed; The feature map has a size of W×H×N, where W represents the width of the feature map, H represents the height of the feature map, and N represents the number of feature channels corresponding to the feature map. The step of dividing the feature information of feature channels belonging to the same image features in the feature map into a feature group includes: The two-dimensional plane of the feature map is divided into at least two planar regions, wherein the two-dimensional plane is represented by W×H, and the number of planar regions is equal to the number of feature groups; Detect the planar region to which the feature information of each feature channel in the feature map belongs; The feature information of feature channels belonging to the same planar region is divided into a feature group; The step of dividing the two-dimensional plane of the feature map into at least two planar regions includes: Identify the structural components of the target object in the image to be processed, and obtain a detection box corresponding to each structural component; Based on the position of the detection box corresponding to each of the structural components, the two-dimensional plane of the feature map is divided into at least two planar regions.

2. The image processing method as described in claim 1, characterized in that, The step of detecting the planar region to which the feature information of each feature channel in the feature map belongs includes: Obtain the target value from the feature information of each feature channel in the feature map; Based on the planar region to which the target value belongs, determine the planar region to which the feature information of each feature channel of the feature map belongs.

3. The image processing method as described in claim 1, characterized in that, The step of generating global features of the image to be processed based on the local features corresponding to each feature group includes: The local features corresponding to each feature group are concatenated to generate a vector, which is the global feature.

4. The image processing method as described in claim 1, characterized in that, The method further includes: The target object in the image to be processed is identified based on the global features, and the identification result is obtained.

5. An image processing apparatus, characterized in that, include: A global extraction unit is used to extract feature information of each of the multiple feature channels in the image to be processed to obtain a feature map, wherein one feature channel corresponds to one image feature; The feature segmentation unit is used to divide the feature information of feature channels belonging to the same image feature in the feature map into a feature group; A local extraction unit is used to extract feature information corresponding to each feature group to obtain local features; A feature generation unit is configured to generate global features of the image to be processed based on the local features corresponding to each feature group, wherein the global features are used to identify target objects in the image to be processed. The feature map has a size of W×H×N, where W represents the width of the feature map, H represents the height of the feature map, and N represents the number of feature channels corresponding to the feature map. Accordingly, the feature division unit is further configured to: divide the two-dimensional plane of the feature map into at least two planar regions, wherein the two-dimensional plane is a plane represented by W×H, and the number of planar regions is equal to the number of feature groups; detect the planar region to which the feature information of each feature channel in the feature map belongs; and divide the feature information of feature channels belonging to the same planar region into a feature group; The feature segmentation unit is further configured to: identify the structural components of the target object in the image to be processed, and obtain a detection box corresponding to each structural component; and divide the two-dimensional plane of the feature map into at least two planar regions according to the position of the detection box corresponding to each structural component.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images