Image processing method, device, computer equipment and computer readable storage medium

Abstract

The embodiments of the present application provide an image processing method, apparatus, computer equipment and computer-readable storage medium, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle networking; the embodiments of the present application can obtain an image to be processed, and cut the image to be processed to obtain at least two image areas; perform feature extraction on the image area to obtain regional feature information of the image area; obtain image positioning information of the image area based on the regional feature information and preset positioning feature information; determine regional association information between image areas based on the regional feature information; fuse the image positioning information and the regional association information to obtain initial fused positioning information; identify the target image area in the image to be processed based on the initial fused positioning information; the embodiments of the present application can comprehensively and accurately identify the target object area of ​​the target object in the image to be processed.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment and computer-readable storage medium. Background Art

[0002] A basic prerequisite task in computer vision analysis is the target localization task, which can determine the location of the target object in the image.

[0003] The current target localization task can use weakly supervised target localization methods to locate the target object in the image. The weakly supervised target localization method generally uses a neural network model for positioning, but this weakly supervised target localization method will cause the neural network model to extract the most discriminative features in the image, so that the neural network model only focuses on the local area of ​​the target object with high discriminability, and cannot fully pay attention to the entire area of ​​the target object.

[0004] In summary, existing target localization tasks cannot fully locate the area of ​​the target object in the image. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, computer device, and computer-readable storage medium, which can comprehensively and accurately identify a target object area in an image to be processed.

[0006] An image processing method, comprising:

[0007] Acquire an image to be processed, and cut the image to be processed to obtain at least two image regions;

[0008] Extracting features from the image region to obtain regional feature information of the image region, where the regional feature information is semantic feature information of the image region;

[0009] According to the region feature information and the preset positioning feature information, image positioning information of the image region is obtained, wherein the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed in each image region;

[0010] Determine regional association information between image regions according to the regional feature information, where the regional association information is information on association between image regions of the image to be processed;

[0011] The image positioning information and the region association information are fused to obtain the initial fused positioning information;

[0012] According to the initial fused positioning information, the target image area in the image to be processed is identified.

[0013] Accordingly, an embodiment of the present application provides an image processing device, comprising:

[0014] A first acquisition unit may be used to acquire an image to be processed and cut the image to be processed to obtain at least two image regions;

[0015] The extraction unit can be used to extract features from the image region to obtain regional feature information of the image region, where the regional feature information is semantic feature information of the image region;

[0016] The second acquisition unit may be used to acquire image positioning information of the image region according to the region feature information and preset positioning feature information, wherein the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed in each image region;

[0017] A determination unit may be used to determine region association information between image regions according to the region feature information, where the region association information is information on association between image regions of the image to be processed;

[0018] A fusion unit can be used to fuse the image positioning information and the region association information to obtain initial fused positioning information;

[0019] The recognition unit can be used to recognize the target image area in the image to be processed according to the initial fusion positioning information.

[0020] In some embodiments, the second acquisition unit can be specifically used to determine the candidate positioning weight of each image region based on the regional feature information and preset positioning feature information; extract the positioning weight of each image region from the candidate positioning weights based on the preset positioning feature information; and use the positioning weight of each image region as the image positioning information of each image region.

[0021] In some embodiments, the second acquisition unit can be specifically used to calculate at least two initial feature information of the image to be processed based on the regional feature information and the preset positioning feature information; and fuse the initial feature information to obtain the candidate positioning weight of each image region.

[0022] In some embodiments, the determination unit can be specifically used to calculate candidate association parameters between image regions based on regional feature information and preset positioning feature information; extract association parameters from candidate association parameters based on regional feature information; and generate regional association information between image regions based on the association parameters.

[0023] In some embodiments, the fusion unit may be specifically configured to perform weighted processing on the region association information according to the image positioning information to obtain weighted positioning information; and determine initial fused positioning information according to the weighted positioning information.

[0024] In some embodiments, the initial fused positioning information includes several initial fused positioning information; the identification unit can be specifically used to fuse the several initial fused positioning information to obtain the target fused positioning information; and identify the target image area in the image to be processed according to the target fused positioning information.

[0025] In some embodiments, the recognition unit may be specifically configured to determine a target positioning image corresponding to the image to be processed according to the positioning information after target fusion; and to identify a target image region in the image to be processed according to the target positioning image.

[0026] In some embodiments, the regional feature information is feature information obtained by extracting features using a trained image processing model; the image processing device also includes a training unit, which can be specifically used to obtain an image sample set, and the image sample set includes at least one image sample annotated with a classification label; the image processing model to be trained is used to encode the image samples to obtain encoded sample feature information; based on the encoded sample feature information, the image samples are type predicted to obtain a predicted classification category of the image samples; based on the predicted classification category and the classification label, the model parameters of the image processing model to be trained are converged to obtain a trained image processing model.

[0027] In addition, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any image processing method provided in the embodiment of the present application.

[0028] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any image processing method provided in the embodiment of the present application.

[0029] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements any image processing method provided in the embodiment of the present application when the computer program is executed by a processor.

[0030] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements any image processing method provided in the embodiment of the present application when the computer program is executed by a processor.

[0031] The embodiment of the present application can obtain an image to be processed, and cut the image to be processed to obtain at least two image regions; extract features from the image region to obtain regional feature information of the image region, where the regional feature is the semantic feature information of the image region; obtain image positioning information of the image region according to the regional feature information and preset positioning feature information, where the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed for each image region; determine regional association information between image regions according to the regional feature information, where the regional association information is information on association between image regions of the image to be processed; fuse the image positioning information and the regional association information to obtain initial fused positioning information; identify the target image region in the image to be processed according to the initial fused positioning information; because the embodiment of the present application can obtain image positioning information according to the regional feature information and the preset positioning feature information, and determine regional association information according to the regional feature information, the target object region in the image to be processed can be fully and accurately identified according to the initial fused positioning information obtained by fusing the image positioning information and the regional association information, that is, the target object region in the image to be processed can be fully and accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 is a scene schematic diagram of the image processing method provided in an embodiment of the present application;

[0034] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application;

[0035] Figure 3 It is a schematic diagram of a process for obtaining image positioning information of an image area according to regional feature information and preset positioning feature information provided in an embodiment of the present application;

[0036] Figure 4 It is a schematic diagram of fusing image positioning information and region association information to obtain initial fused positioning information provided by an embodiment of the present application;

[0037] Figure 5 FIG2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0038] Figure 6A schematic diagram of the structure of the image processing model to be trained provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the structure of the encoding module provided in the embodiment of the present application;

[0040] Figure 8 Three diagrams showing the process of the image processing method provided in the embodiments of the present application;

[0041] Fig. 9 A schematic diagram of a process for determining a first attention image corresponding to each image area provided in an embodiment of the present application;

[0042] Fig.10 The image positioning information, region association information, target positioning image, and visualization diagram of the target object region provided in the embodiments of the present application;

[0043] Fig.11 A schematic diagram comparing the existing target object area provided in the embodiment of the present application and the target object area of ​​the present application;

[0044] Fig.12 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0045] Fig.13 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0047] The embodiments of the present application provide an image processing method, apparatus, computer device and computer-readable storage medium. The image processing apparatus can be integrated in a computer device, which can be a server or a terminal.

[0048] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0049] Among them, the embodiments of the present application involve artificial intelligence (AI), which is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0050] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0051] For example, see Figure 1 Taking the image processing device integrated in a computer device as an example, the computer device obtains an image to be processed, and cuts the image to be processed to obtain at least two image regions; extracts features from the image region to obtain regional feature information of the image region; obtains image positioning information of the image region based on the regional feature information and preset positioning feature information; determines regional association information between image regions based on the regional feature information; fuses the image positioning information and the regional association information to obtain initial fused positioning information; and identifies the target image region in the image to be processed based on the initial fused positioning information.

[0052] The image to be processed may be any image, for example, the image to be processed may be a landscape image, a portrait image, an object image, and so on.

[0053] The preset positioning feature information is preset feature information for positioning the target object in the image to be processed. The target object may refer to the object to be positioned in the image to be processed. Specifically, the preset positioning feature information may be feature information that is pre-set, and the preset positioning feature information may also be feature information for initializing a neural network model.

[0054] The region feature information may refer to information representing the features of the image region. Specifically, the region feature information is semantic feature information of the image region.

[0055] Among them, the image positioning information is the information of each image area for positioning the target object in the image to be processed. The image positioning information can be specifically expressed as the contribution of each image area to positioning the target object in the image to be processed. For example, the image positioning information is expressed as the positioning weight of each image area for positioning the target object in the image to be processed. The representation form of the image positioning information can be a numerical value.

[0056] The region association information is information about the association between image regions of the image to be processed, and the representation form of the region association information may be a numerical value.

[0057] The initial fused positioning information may refer to information obtained by fusing the image positioning information and the region association information.

[0058] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0059] This embodiment will be described from the perspective of an image processing device, which can be specifically integrated in a computer device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can acquire data.

[0060] First of all, it should be noted that in the existing related technologies, most of the existing weakly supervised target positioning methods are based on the convolutional neural network model to locate the target object in the image to be processed. Specifically, the convolutional neural network model first uses the backbone network to extract the deep semantic features of the image to be processed, and then globally pools the deep semantic features, and then classifies the image to be processed through the fully connected layer to obtain the target classification category, and then calculates the loss value of the convolutional neural network model through the cross entropy loss function. In this way, the convolutional neural network model automatically pays attention to the target object corresponding to the target classification category in the image to be processed, and pays more attention to the local area of ​​the target object with higher discrimination in the image to be processed, and cannot fully pay attention to the entire area of ​​the target object in the image to be processed. In this way, it is impossible to fully and accurately locate the area of ​​the target object in the image to be processed, resulting in incomplete or empty detected target objects.

[0061] Based on the above, the embodiment of the present application provides an image processing method, which can comprehensively and accurately identify the target object area of ​​the target object in the image to be processed, such as Figure 2 As shown, the specific process of the image processing method is shown in S101 to S106:

[0062] S101, obtaining an image to be processed, and cutting the image to be processed to obtain at least two image regions.

[0063] The image to be processed may be any image, for example, the image to be processed may be a landscape image, a portrait image, an object image, and so on.

[0064] Among them, the size of each image area in the embodiment of the present application can be the same.

[0065] In the embodiment of the present application, there are multiple ways to cut the image to be processed to obtain at least two image regions, as follows:

[0066] For example, the computer device uses a neural network model to segment the image to be processed to obtain at least two image regions.

[0067] For another example, the computer device cuts the image to be processed according to a preset region size to obtain at least two image regions, wherein the preset region size is a parameter pre-set to be used as the size of the image region.

[0068] S102: Extract features from the image region to obtain regional feature information of the image region.

[0069] The region feature information may refer to information representing the features of the image region. Specifically, the region feature information is semantic feature information of the image region.

[0070] Specifically, the computer device can extract semantic features of the image region to obtain regional feature information of the image region.

[0071] In the embodiment of the present application, there are multiple ways to extract features from an image region and obtain regional feature information of the image region, as follows:

[0072] For example, the embodiments of the present application may use a neural network model in existing related technologies to extract features from an image region to obtain regional feature information of the image region.

[0073] For another example, the embodiments of the present application may use a trained image processing model to extract features from an image region to obtain regional feature information of the image region.

[0074] The trained image processing model may be a model obtained by training the image processing model to be trained. The embodiment of the present application may train the image processing model to be trained as follows:

[0075] For example, a computer device can obtain an image sample set, which includes at least one image sample marked with a classification label; encode the image samples using the image processing model to be trained to obtain feature information of the encoded samples; predict the type of the image samples based on the feature information of the encoded samples to obtain a predicted classification category of the image samples; converge the model parameters of the image processing model to be trained based on the predicted classification category and the classification label to obtain a trained image processing model.

[0076] The image processing model to be trained in the embodiment of the present application may be a model using Transformer as the backbone network, and Transformer has an encoder-decoder structure. Based on this, the embodiment of the present application uses the image processing model to be trained to encode the image sample, and the encoded sample feature information may be specifically: the computer device uses the encoder of the image processing model to be trained to encode the image sample, and obtains the encoded sample feature information.

[0077] The image processing model to be trained in the embodiment of the present application uses Transformer as the backbone network. Compared with convolutional neural networks, Transformer not only has powerful feature modeling capabilities, but also has excellent global feature extraction capabilities. Transformer relies on a simple yet powerful attention mechanism, which enables the image processing model to be trained to fully correlate and analyze and model all areas in the image sample. Transformer can effectively model the global features of the image sample through its internal encoder-decoder structure. By using Transformer as the backbone network, the embodiment of the present application can fully pay attention to all areas of the target object in the image sample.

[0078] S103: Obtain image positioning information of the image area according to the area feature information and preset positioning feature information.

[0079] The preset positioning feature information is preset feature information for positioning the target object in the image to be processed. The target object may refer to the object to be positioned in the image to be processed. Specifically, the preset positioning feature information may be feature information that is pre-set, and the preset positioning feature information may also be feature information for initializing a neural network model.

[0080] Among them, the image positioning information is the information of each image area for positioning the target object in the image to be processed. The image positioning information can be specifically expressed as the contribution of each image area to positioning the target object in the image to be processed. For example, the image positioning information is expressed as the positioning weight of each image area for positioning the target object in the image to be processed. The representation form of the image positioning information can be a numerical value.

[0081] like Figure 3 As shown, in the embodiment of the present application, the method of obtaining the image positioning information of the image area according to the regional feature information and the preset positioning feature information can be as shown in A1 to A3:

[0082] A1. Determine the candidate positioning weight of each image region according to the region feature information and the preset positioning feature information.

[0083] In the embodiment of the present application, there are multiple ways to determine the candidate positioning weight of each image region according to the regional feature information and the preset positioning feature information, as follows:

[0084] For example, the preset positioning feature information includes several preset positioning feature information; the computer device can calculate the target similarity between the regional feature information and the preset positioning feature information to obtain the target similarity of each image area for the preset positioning feature information; for each image area, the target similarity corresponding to the image area is normalized to obtain the candidate positioning weight of each image area.

[0085] For another example, the computer device may calculate at least two initial feature information of the image to be processed based on the regional feature information and the preset positioning feature information; and perform fusion processing on the initial feature information to obtain the candidate positioning weight of each image region.

[0086] Among them, the embodiment of the present application can calculate at least two initial feature information of the image to be processed based on the regional feature information and the preset positioning feature information. The method can be: the computer device obtains at least two mapping functions; for each mapping function, each mapping function is used to map the regional feature information and the preset positioning feature information to obtain the initial feature information corresponding to each mapping function, and obtain at least two initial feature information of the image to be processed.

[0087] Among them, the initial feature information can be represented in the form of a matrix. Based on this, the embodiment of the present application performs a fusion process on the initial feature information to obtain the candidate positioning weight of each image area: the computer device calculates the average value of at least two initial feature information to obtain the candidate positioning weight of each image area.

[0088] A2. According to the preset positioning feature information, the positioning weight of each image area is extracted from the candidate positioning weights.

[0089] Among them, the preset positioning feature information can have a corresponding relationship with the regional feature information corresponding to the image area, and the candidate positioning weight is the weight obtained after processing the regional feature information and the preset positioning feature information. Based on this, the embodiment of the present application can preset the positioning feature information and extract the positioning weight corresponding to each image area from the candidate positioning weight.

[0090] A3. Using the positioning weight of each image region as the image positioning information of each image region.

[0091] In order to more accurately determine the target object region of the target object in the image to be processed, the embodiment of the present application can also locate the target object region in the image to be processed in combination with the region association information between the image regions.

[0092] S104: Determine region association information between image regions according to the region feature information.

[0093] The region association information may be information representing the association between image regions in the image to be processed, and the representation form of the region association information may be a matrix.

[0094] In the embodiment of the present application, the method of determining the regional association information between image regions according to the regional feature information may be as follows:

[0095] For example, the computer device can calculate candidate association parameters between image regions based on regional feature information and preset positioning feature information; extract association parameters from candidate association parameters based on the regional feature information; and generate regional association information between image regions based on the association parameters.

[0096] Among them, the way in which the computer device calculates the candidate association parameters between image regions based on the regional feature information and the preset positioning feature information can be: the computer device can calculate at least two reference feature information of the image to be processed based on the regional feature information and the preset positioning feature information; and fuse the reference feature information to obtain the candidate association parameters between the image regions.

[0097] The reference feature information can be represented in the form of a matrix. Based on this, the embodiment of the present application performs fusion processing on the reference feature information to obtain candidate association parameters between image areas: a computer device calculates the average value of at least two reference feature information to obtain candidate association parameters between image areas.

[0098] S105: Fusing the image positioning information and the region association information to obtain initial fused positioning information.

[0099] The embodiment of the present application obtains the outgoing fused positioning information by fusing the image positioning information and the region association information, which can more comprehensively and accurately identify the target object area in the image to be processed. The initial fused positioning information can refer to the information obtained by fusing the image positioning information and the region association information.

[0100] Specifically, the image positioning information includes the positioning weight corresponding to each image area. Based on this, the embodiment of the present application can fuse the image positioning information and the area association information in the following manner: the computer device can perform weighted processing on the area association information according to the image positioning information to obtain weighted positioning information; and determine the initial fused positioning information based on the weighted positioning information.

[0101] S106: Identify the target image area in the image to be processed according to the initial fused positioning information.

[0102] The target image region refers to the image region of the target object in the image to be processed.

[0103] The initial fused positioning information includes several initial fused positioning information, based on which, such as Figure 4 As shown, in the embodiment of the present application, the image positioning information and the region association information are fused and processed, and the method of obtaining the initial fused positioning information can be as follows: steps B1 to B2:

[0104] B1. Fuse several initial fused positioning information to obtain target fused positioning information.

[0105] Among them, the initial fused positioning information can be represented in the form of a matrix. Based on this, the embodiment of the present application fuses several initial fused positioning information to obtain the target fused positioning information in the following way: the computer device can calculate the initial average value of several initial fused positioning information, and the initial average value is the target fused positioning information.

[0106] B2. According to the positioning information after target fusion, the target image area in the image to be processed is identified.

[0107] In the embodiment of the present application, the method of identifying the target image area in the image to be processed according to the positioning information after the target fusion can be as follows:

[0108] For example, the computer device can determine the target positioning image corresponding to the image to be processed based on the positioning information after the target fusion; and identify the target image area in the image to be processed based on the target positioning image.

[0109] The target fusion positioning information can be represented in the form of a matrix. In the embodiment of the present application, the target fusion positioning information can be used to determine the target positioning image corresponding to the image to be processed by performing a matrix transformation operation on the target fusion positioning information to obtain the target positioning image corresponding to the image to be processed. The target positioning image can be an activation map, specifically a thermal activation map.

[0110] Among them, the embodiment of the present application can perform binarization processing on the target positioning image to obtain a binarized positioning image; and adopt a connected region analysis strategy for the binarized positioning image to identify the target image region in the image to be processed.

[0111] The connected domain analysis strategy can be a Two-Pass strategy or a Seed-Filling strategy.

[0112] The embodiment of the present application can obtain an image to be processed, and cut the image to be processed to obtain at least two image regions; extract features from the image region to obtain regional feature information of the image region, where the regional feature is the semantic feature information of the image region; obtain image positioning information of the image region according to the regional feature information and preset positioning feature information, where the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed for each image region; determine regional association information between image regions according to the regional feature information, where the regional association information is information on association between image regions of the image to be processed; fuse the image positioning information and the regional association information to obtain initial fused positioning information; identify the target image region in the image to be processed according to the initial fused positioning information; because the embodiment of the present application can obtain image positioning information according to the regional feature information and the preset positioning feature information, and determine regional association information according to the regional feature information, the target object region in the image to be processed can be fully and accurately identified according to the initial fused positioning information obtained by fusing the image positioning information and the regional association information, that is, the target object region in the image to be processed can be fully and accurately identified.

[0113] According to the method described in the above embodiment, the following is further described in detail with examples.

[0114] In this embodiment, the image processing apparatus is specifically integrated into a computer device, and the computer device is a server.

[0115] like Figure 5 As shown, an image processing method, the specific process is as follows: step S201 to step S210:

[0116] S201. A computer device obtains a set of image samples.

[0117] The image sample set includes at least one image sample annotated with a classification label.

[0118] S202. The computer device encodes the image samples using the image processing model to be trained to obtain feature information of the encoded samples.

[0119] Among them, the image processing model to be trained in the embodiment of the present application can be a model that uses Transformer as the backbone network, and there is an encoder-decoder structure in Transformer.

[0120] In an embodiment of the present application, the image processing model to be trained may be a ViT model. The ViT model adopts the Transformer structure in the BERT neural network.

[0121] The embodiment of the present application uses the encoder in the Transformer structure of the ViT model for encoding to obtain encoded sample feature information.

[0122] The image processing model to be trained is as follows Figure 6 As shown, the image processing model to be trained includes a feature extraction layer 11, a Transformer encoding layer 12 and a classification layer 13. In the embodiment of the present application, the computer device encodes the image sample using the image processing model to be trained, and the process of obtaining the encoded sample feature information can be specifically as follows:

[0123] The computer device cuts the image sample to obtain a number of image sample areas; the feature extraction layer 11 of the image processing model to be trained is used to extract features from each image sample area to obtain sample area feature information; the sample area feature information and the preset classification feature information are spliced ​​to obtain spliced ​​sample feature information; the Transformer encoding layer 12 of the image processing model to be trained is used to encode the spliced ​​sample feature information to obtain encoded sample feature information.

[0124] Among them, the preset classification feature information is the feature information randomly initialized by the ViT model.

[0125] Among them, the post-splicing sample feature information includes sample area feature information and preset classification feature information; before encoding the post-splicing sample feature information, the implementation of this application can also obtain position feature information.

[0126] Based on the above, an embodiment of the present application encodes the spliced ​​sample feature information, and a method for obtaining the encoded sample feature information may specifically include: a computer device merges the position feature information with the sample area feature information and the preset classification feature information in the spliced ​​sample feature information one by one to obtain target sample feature information, wherein the target sample feature information includes sample area feature information carrying position feature information and preset classification feature information carrying position feature information; and uses the encoding layer in the Transformer structure of the ViT model to encode the target sample feature information to obtain the encoded sample feature information.

[0127] Specifically, the Transformer encoding layer of the image processing model to be trained includes at least one encoding module (block), such as Figure 7 As shown, the encoding module includes a first normalization layer (first Norm layer), a multi-head attention layer (Multi-Head Attention layer), a second normalization layer (second Norm layer) and an output layer (MLP layer).

[0128] In the embodiment of the present application, the target sample feature information is input into the first standardization layer through the image processing model to be trained; the first standardization layer is used to extract the target sample feature information to obtain the first sample feature information; the multi-head attention layer is used to extract the first sample feature information to obtain the initial sample feature information; the initial sample feature information and the target sample feature information are fused to obtain the first fused sample feature information; the second standardization layer is used to extract the first fused sample feature information to obtain the second sample feature information; the output layer is used to extract the second sample feature information to obtain the third sample feature information; the third sample feature information and the first fused sample feature information are fused to obtain the second fused sample feature information. Wherein, the second fused sample feature information is the above-mentioned encoded sample feature information.

[0129] When the Transformer encoding layer of the embodiment of the present application has multiple encoding modules, the encoding modules are connected in series in sequence. For example, the encoding module includes a first encoding module and a second encoding module, the first encoding module and the second encoding module are connected in series, and the feature information encoded by the first encoding module is input to the second encoding module for encoding. The feature information output by the last encoding module is the above-mentioned encoded sample feature information.

[0130] S203: The computer device predicts the type of the image sample according to the encoded sample feature information to obtain a predicted classification category of the image sample.

[0131] Based on the above, the embodiment of the present application predicts the type of the image sample according to the encoded sample feature information, and obtains the predicted classification category of the image sample in the following manner: Figure 6 As shown, the computer device classifies the image samples according to the encoded sample feature information using the classification layer 13 of the image processing model to be trained to obtain the predicted classification category of the image samples.

[0132] S204. The computer device converges the model parameters of the image processing model to be trained according to the predicted classification category and the classification label to obtain a trained image processing model.

[0133] Among them, the embodiment of the present application adopts a loss function to calculate the loss value between the predicted classification category and the classification label; the model parameters of the image processing model to be trained are converged according to the loss value to obtain the trained image processing model.

[0134] S205: The computer device obtains the image to be processed, and cuts the image to be processed to obtain at least two image regions.

[0135] In the embodiment of the present application, the method of cutting the image to be processed to obtain at least two image regions may be as follows:

[0136] For example, the computer device uses the trained image processing model to segment the image to be processed to obtain at least two image regions.

[0137] The size of each image area may be the same.

[0138] S206. The computer device uses the trained image processing model to extract features from the image region to obtain regional feature information of the image region.

[0139] In this embodiment of the present application, the trained image processing model can be used to extract features from each image region to obtain regional feature information of each image region. In this embodiment of the present application, the regional feature information of each image region can be represented by a one-dimensional matrix through dimensional conversion.

[0140] S207: The computer device obtains image positioning information of the image area according to the area feature information and preset positioning feature information.

[0141] Among them, the preset positioning feature information in the embodiment of the present application is the feature information initialized by the trained image processing model.

[0142] like Figure 3 As shown, in the embodiment of the present application, the method of obtaining the image positioning information of the image area according to the regional feature information and the preset positioning feature information can be as shown in A1 to A3:

[0143] A1. Determine the candidate positioning weight of each image region according to the region feature information and the preset positioning feature information.

[0144] In the embodiment of the present application, the method of determining the candidate positioning weight of each image region according to the regional feature information and the preset positioning feature information can be as follows:

[0145] For example, the computer device can calculate at least two initial feature information of the image to be processed based on the regional feature information and the preset positioning feature information; and perform fusion processing on the initial feature information to obtain the candidate positioning weight of each image region.

[0146] Among them, the initial feature information can be represented in the form of a matrix. Based on this, the embodiment of the present application performs a fusion process on the initial feature information to obtain the candidate positioning weight of each image area: the computer device calculates the average value of at least two initial feature information to obtain the candidate positioning weight of each image area.

[0147] Among them, the trained image processing model of the embodiment of the present application can obtain special input, namely, target position feature information. This is because the Transformer and CNN of the trained image processing model are different, and the Transformer needs to use the target position feature information to encode the relative position information of the regional feature information and the relative position information of the preset positioning feature information.

[0148] This is mainly because the encoding layer of Transformer is insensitive to the order of the input regional feature information and the preset positioning feature information, that is, disrupting the order of the regional feature information and / or the preset positioning feature information in the sequence will not change the output result of the encoding layer.

[0149] If the relative position information of the regional feature information and the preset positioning feature information is not actively provided to the trained image processing model, then the trained image processing model will need to learn to infer the relative position relationship between the regional feature information and / or the preset positioning feature information, which will increase the learning cost and result in a decrease in the reasoning accuracy of the trained image processing model.

[0150] Based on the above, the embodiment of the present application can calculate at least two initial feature information of the image to be processed according to the regional feature information and the preset positioning feature information in a manner as follows: a computer device obtains the target position feature information; splices the regional feature information and the preset positioning feature information to obtain the target spliced ​​feature information; fuses the target position feature information and the target spliced ​​feature information to obtain the fused feature information, wherein the target position feature information and the sub-feature information in the target spliced ​​feature information correspond one to one, and the sub-feature information includes the regional feature information and the preset positioning feature information; and determines at least two initial feature information of the image to be processed according to the fused feature information, wherein the fused feature information may include the preset positioning feature information carrying the target position feature information and the regional feature information carrying the target position feature information.

[0151] Specifically, the Transformer encoding layer of the trained image processing model includes at least one encoding module, which includes a first normalization layer (first Norm layer), a multi-head attention layer (Multi-Head Attention layer), a second normalization layer (second Norm layer) and an output layer (MLP layer). For details, please refer to the above Figure 7 The explanation in will not be repeated here.

[0152] In the embodiment of the present application, the fused feature information is input into the first normalization layer; the first normalization layer is used to perform feature extraction on the fused feature information to obtain first feature information; and the multi-head attention layer is used to perform feature extraction on the first feature information to obtain at least two initial feature information of the image to be processed.

[0153] Specifically, the fused feature information can be expressed as X l ∈R (N+1)×D , where R (N+1)×D It can be a matrix with a dimension of (N+1)×D, where N+1 represents N image regions and 1 preset positioning feature information, and D can represent the number of elements in each image region and the number of elements in the preset positioning feature information.

[0154] The multi-head attention layer of the embodiment of the present application has at least two attention heads. After training, the image processing model can calculate the initial feature information corresponding to each attention head based on the fused feature information. The representation of the initial feature information can be shown as formula (1):

[0155]

[0156] Among them, A l represents the initial feature information, A l ∈R S×(N+1)×(N+1) ;Q l Indicates the query value (Queries) mapped by the fused feature information; K l Τ represents the transposition of the key value (Key) mapped by the fused feature information; S represents the number of attention heads; D can represent the number of elements in each image area and the number of elements in the preset positioning feature information.

[0157] In this embodiment of the present application, the query value and key value of each attention head may be different.

[0158] Based on the above, the embodiment of the present application performs fusion processing on the initial feature information to obtain the candidate positioning weight of each image area as follows:

[0159] For each encoding module, the computer device can calculate the average value of the initial feature information corresponding to all attention heads to obtain a candidate positioning weight set, where the candidate positioning weight set includes the candidate positioning weight corresponding to each image region. The candidate positioning weight set can be expressed as A l '∈R (N+1)×(N+1) .

[0160] A2. According to the preset positioning feature information, the positioning weight of each image area is extracted from the candidate positioning weights.

[0161] Among them, the preset positioning feature information can have a corresponding relationship with the regional feature information corresponding to the image area, and the candidate positioning weight is the weight obtained after processing the regional feature information and the preset positioning feature information. Based on this, the embodiment of the present application can preset the positioning feature information and extract the positioning weight corresponding to each image area from the candidate positioning weight.

[0162] Based on the above, the embodiment of the present application can extract the positioning weight of each image area from the candidate positioning weight in the candidate positioning weight set. The positioning weight of each image area can be expressed as It reflects the contribution of each image region to the positioning of the target object in the image to be processed. It can be regarded as the response degree of the image processing model to each image region. The positioning weight of each image region can refer to the positioning weight of each image region corresponding to the preset positioning feature information, that is, The association between the preset positioning feature information and each image area is modeled.

[0163] A3. Using the positioning weight of each image region as the image positioning information of each image region.

[0164] Based on the above, the image positioning information can be expressed as

[0165] However, in the embodiment of the present application, it is not enough to simply model the association between the preset positioning feature information and each image area. This ignores the correlation between image areas and image areas, resulting in weak local image response, which is not conducive to positioning the target object of the image to be processed.

[0166] Based on this, the embodiment of the present application introduces image regions and association information between image regions to supplement and assist in locating the target object of the image to be processed.

[0167] S208: The computer device determines region association information between image regions according to the region feature information.

[0168] The region association information may be information representing the association between image regions in the image to be processed, and the representation form of the region association information may be a numerical value.

[0169] In the embodiment of the present application, the method of determining the regional association information between image regions according to the regional feature information may be as follows:

[0170] For example, the computer device can calculate candidate association parameters between image regions based on regional feature information and preset positioning feature information; extract association parameters from candidate association parameters based on the regional feature information; and generate regional association information between image regions based on the association parameters.

[0171] Specifically, the candidate association parameters in the embodiment of the present application constitute a candidate association parameter set, and the candidate association parameter set may be the candidate positioning weight set A mentioned above. l '∈R (N+1)×(N+1) In the embodiment of the present application, the association parameters are extracted from the candidate association parameters according to the regional feature information. The association parameters can constitute the regional association information. The regional association information can be expressed as M l p ∈R (N+1)×N .

[0172] The regional correlation information expresses the degree of correlation between different image regions and can well model the local correlation of the image to be processed.

[0173] S209: The computer device fuses the image positioning information and the region association information to obtain initial fused positioning information.

[0174] In order to comprehensively and accurately identify the target object area, the computer device in the embodiment of the present application combines the image positioning information and the area association information.

[0175] Among them, the image positioning information includes the positioning weight corresponding to each image area. Based on this, the embodiment of the present application can fuse the image positioning information and the area association information in the following way: weight the area association information according to the image positioning information to obtain weighted positioning information; and determine the initial fused positioning information based on the weighted positioning information.

[0176] Specifically, the embodiment of the present application is based on the image positioning information For regional association information M l p ∈R (N +1)×N Perform weighted processing to obtain weighted positioning information M l h ∈R (N+1)×N ; Then, the weighted positioning information Average to get the initial fusion positioning information Among them, the embodiment of the present application can be based on the weighted positioning information M l h ∈R (N+1)×N The elements of each column in are averaged to obtain the initial fusion positioning information

[0177] S210: The computer device identifies a target image region in the image to be processed according to the initial fused positioning information.

[0178] In the embodiment of the present application, each encoding module of the trained image processing model corresponds to the initial fusion positioning information That is, the initial fused positioning information includes a plurality of initial fused positioning information. Based on this, the embodiment of the present application fuses the image positioning information and the region association information to obtain the initial fused positioning information in the following manner:

[0179] For example, the computer device may fuse a number of initial fused positioning information to obtain target fused positioning information; and identify the target image area in the image to be processed based on the target fused positioning information.

[0180] Among them, the initial fusion positioning information can be represented in the form of a matrix, that is, the initial fusion positioning information can be represented as Based on this, the embodiment of the present application fuses several initial fused positioning information to obtain target fused positioning information in the following manner: the computer device can calculate the initial average value of several initial fused positioning information, and the initial average value is the target fused positioning information.

[0181] Specifically, the computer device calculates some initial fusion positioning information The initial average value of the target fusion positioning information is represented as

[0182] Based on this, the embodiment of the present application can identify the target image area in the image to be processed according to the positioning information after the target is fused as follows:

[0183] For example, the computer device can determine the target positioning image corresponding to the image to be processed based on the positioning information after the target fusion; and identify the target image area in the image to be processed based on the target positioning image.

[0184] The target fusion positioning information can be represented in the form of a matrix. In the embodiment of the present application, the target fusion positioning information can be used to determine the target positioning image corresponding to the image to be processed by performing a matrix transformation operation on the target fusion positioning information to obtain the target positioning image corresponding to the image to be processed. The target positioning image can be an activation map, specifically a thermal activation map.

[0185] Specifically, the target positioning image can be expressed as formula (2):

[0186]

[0187] in, represents the target positioning image; Γ w×h Represents a matrix transformation operation.

[0188] Among them, the target positioning image can be represented in the form of a two-dimensional matrix in the vector space.

[0189] Furthermore, specifically, Figure 8 As shown in , the image positioning information can be represented by a matrix, based on which, the image positioning information includes the positioning weight corresponding to each image region. The region association information can be represented in the form of a matrix, and the elements in the matrix corresponding to the region association information include the association parameters corresponding to each image region. The association parameters corresponding to each image region are visualized as the first attention image. The first attention image is as shown in Figure 8 shown.

[0190] The embodiment of the present application performs weighted processing on the positioning weight of the image positioning information for the attention image corresponding to each image region to obtain weighted positioning information. The weighted positioning information can be represented by a matrix, based on which the weighted positioning information includes elements corresponding to each image region. The elements corresponding to each image region are visualized as the second attention image such as Figure 8 shown.

[0191] Furthermore, if Fig. 9 As shown, the embodiment of the present application cuts the image to be processed to obtain at least two image regions, wherein there are n image regions, and n is a positive integer greater than 2; and extracts features from the image regions to obtain regional feature information of the image regions, wherein: Fig. 9 Medium α n = represents the region feature information corresponding to the nth image region. According to the region feature information and the preset positioning feature information, the region association information is determined. The determination of the region association information here can be referred to above and will not be repeated here. Since the region association information includes the association parameters corresponding to each image region, the association parameters corresponding to each image region are visualized as an attention image. The first attention image corresponding to each image region is as follows: Fig. 9 As shown, there are n image regions and n corresponding first attention images.

[0192] Based on the above, the embodiment of the present application can determine the initial fused positioning information based on the weighted positioning information; determine the target positioning image based on the initial fused positioning information; the specific process of determining the initial fused positioning information based on the weighted positioning information; and determining the target positioning image based on the initial fused positioning information can refer to the specific content of determining the target positioning image corresponding to the image to be processed mentioned above, which will not be repeated here.

[0193] Among them, the embodiment of the present application can perform binarization processing on the target positioning image to obtain a binarized positioning image; and adopt a connected region analysis strategy for the binarized positioning image to identify the target image region in the image to be processed.

[0194] The connected domain analysis strategy can be a Two-Pass strategy or a Seed-Filling strategy.

[0195] Based on the above, the embodiment of the present application further explains that the embodiment of the present application can visualize image positioning information, region association information, target positioning image, and target object region, such as Fig.10 shown.

[0196] In the embodiment of the present application, the prior art recognizes the image to be processed to obtain the existing target object area, and the embodiment of the present application recognizes the image to be processed to obtain the target object area of ​​the present application. Fig.11 As shown, Fig.11 The area in the box is the target object area. It can be clearly seen that the target object area of ​​the present application is obviously more comprehensive and accurate than the existing target object area, and there is no problem of incomplete recognition.

[0197] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0198] The embodiment of the present application can obtain an image to be processed, and cut the image to be processed to obtain at least two image regions; extract features from the image region to obtain regional feature information of the image region, where the regional feature is the semantic feature information of the image region; obtain image positioning information of the image region according to the regional feature information and preset positioning feature information, where the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed for each image region; determine regional association information between image regions according to the regional feature information, where the regional association information is information on association between image regions of the image to be processed; fuse the image positioning information and the regional association information to obtain initial fused positioning information; identify the target image region in the image to be processed according to the initial fused positioning information; because the embodiment of the present application can obtain image positioning information according to the regional feature information and the preset positioning feature information, and determine regional association information according to the regional feature information, the target object region in the image to be processed can be fully and accurately identified according to the initial fused positioning information obtained by fusing the image positioning information and the regional association information, that is, the target object region in the image to be processed can be fully and accurately identified.

[0199] In order to better implement the above method, an embodiment of the present application also provides an image processing device, which can be integrated into a computer device, such as a server or a terminal. The terminal may include a tablet computer, a laptop computer and / or a personal computer.

[0200] For example, Fig.12 As shown, the image processing device may include a first acquisition unit 301, an extraction unit 302, a second acquisition unit 303, a determination unit 304, a fusion unit 305, a recognition unit 306 and a training unit 307, as follows:

[0201] (1) a first acquisition unit 301;

[0202] The first acquisition unit 301 may be used to acquire the image to be processed and cut the image to be processed to obtain at least two image regions.

[0203] (2) Extraction unit 302;

[0204] The extraction unit 302 may be used to extract features from the image region to obtain region feature information of the image region, where the region feature information is semantic feature information of the image region.

[0205] (3) a second acquisition unit 303;

[0206] The second acquisition unit 303 can be used to obtain image positioning information of the image area based on the regional feature information and preset positioning feature information. The preset positioning feature information is preset feature information for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed in each image area.

[0207] In some embodiments, the second acquisition unit 303 can be used to determine the candidate positioning weight of each image region based on regional feature information and preset positioning feature information; extract the positioning weight of each image region from the candidate positioning weights based on the preset positioning feature information; and use the positioning weight of each image region as the image positioning information of each image region.

[0208] In some embodiments, the second acquisition unit 303 can be used to calculate at least two initial feature information of the image to be processed based on the regional feature information and the preset positioning feature information; and perform fusion processing on the initial feature information to obtain the candidate positioning weight of each image region.

[0209] (4) determining unit 304;

[0210] The determination unit 304 may be configured to determine region association information between image regions according to the region feature information, where the region association information is information on association between image regions of the image to be processed.

[0211] In some embodiments, the determination unit 304 can be used to calculate candidate association parameters between image regions based on regional feature information and preset positioning feature information; extract association parameters from candidate association parameters based on regional feature information; and generate regional association information between image regions based on the association parameters.

[0212] (5) fusion unit 305;

[0213] The fusion unit 305 may be used to fuse the image positioning information and the region association information to obtain initial fused positioning information.

[0214] In some embodiments, the fusion unit 305 may be specifically configured to perform weighted processing on the region association information according to the image positioning information to obtain weighted positioning information; and determine initial fused positioning information according to the weighted positioning information.

[0215] (6) identification unit 306;

[0216] The identification unit 306 may be used to identify a target image region in the image to be processed according to the initial fused positioning information.

[0217] In some embodiments, the initial fused positioning information includes several initial fused positioning information; the identification unit 306 can be specifically used to fuse the several initial fused positioning information to obtain the target fused positioning information; and identify the target image area in the image to be processed according to the target fused positioning information.

[0218] In some embodiments, the identification unit 306 may be specifically configured to determine a target positioning image corresponding to the image to be processed according to the positioning information after target fusion; and identify a target image region in the image to be processed according to the target positioning image.

[0219] (7) training unit 307;

[0220] The regional feature information is feature information obtained by extracting features using a trained image processing model; the training unit 307 can be used to obtain an image sample set, which includes at least one image sample annotated with a classification label; the image sample is encoded using the image processing model to be trained to obtain encoded sample feature information; based on the encoded sample feature information, the image sample is type predicted to obtain a predicted classification category of the image sample; based on the predicted classification category and the classification label, the model parameters of the image processing model to be trained are converged to obtain a trained image processing model.

[0221] As can be seen from the above, the first acquisition unit 301 of the embodiment of the present application can be used to acquire the image to be processed, and cut the image to be processed to obtain at least two image regions; the extraction unit 302 can be used to extract features of the image region to obtain regional feature information of the image region, and the regional feature is the semantic feature information of the image region; the second acquisition unit 303 can be used to acquire image positioning information of the image region according to the regional feature information and preset positioning feature information, the preset positioning feature information is the preset feature information for locating the target object in the image to be processed, and the image positioning information is the information for locating the target object in the image to be processed for each image region; the determination unit 304 can be used to determine the regional association information between the image regions according to the regional feature information. The regional association information is the information on the association between the image regions of the image to be processed; the fusion unit 305 can be used to fuse the image positioning information and the regional association information to obtain initial fused positioning information; the identification unit 306 can be used to identify the target image region in the image to be processed according to the initial fused positioning information; since the embodiment of the present application can obtain the image positioning information according to the regional feature information and the preset positioning feature information, and determine the regional association information according to the regional feature information, the target object region in the image to be processed can be comprehensively and accurately identified according to the initial fused positioning information obtained by fusing the image positioning information and the regional association information, that is, the target object region in the image to be processed can be comprehensively and accurately identified.

[0222] The present application also provides a computer device, such as Fig.13 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0223] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.13 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0224] The processor 401 is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the computer device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0225] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0226] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0227] The computer device may further include an input unit 404, which may be used to receive input digital or character information communications and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0228] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer programs stored in the memory 402, thereby realizing various functions, as follows:

[0229] Acquire an image to be processed, and cut the image to be processed to obtain at least two image regions; perform feature extraction on the image region to obtain region feature information of the image region, wherein the region feature information is semantic feature information of the image region; acquire image positioning information of the image region according to the region feature information and preset positioning feature information, wherein the preset positioning feature information is feature information preset for positioning a target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed by each image region; determine region association information between the image regions according to the region feature information, wherein the region association information is information on association between image regions of the image to be processed; fuse the image positioning information and the region association information to obtain initial fused positioning information; and identify the target image region in the image to be processed according to the initial fused positioning information.

[0230] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0231] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0232] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any image processing method provided in the embodiment of the present application.

[0233] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0234] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0235] Since the instructions stored in the computer-readable storage medium can execute the steps in any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0236] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0237] The above is a detailed introduction to an image processing method, computer equipment, device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed, and cut the image to be processed to obtain at least two image regions; Extracting features from the image region to obtain regional feature information of the image region, wherein the regional feature information is semantic feature information of the image region; According to the region feature information and preset positioning feature information, image positioning information of the image region is obtained, wherein the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed by each image region, including: Determining a candidate positioning weight for each image region according to the region feature information and preset positioning feature information; According to the preset positioning feature information, extracting the positioning weight of each image area from the candidate positioning weights, the positioning weight reflecting the contribution of the corresponding image area to the positioning of the target object in the image to be processed; Using the positioning weight of each image region as image positioning information of each image region; Determining, according to the region feature information, region association information between the image regions, wherein the region association information is information on association between the image regions of the image to be processed; Fusing the image positioning information and the region association information to obtain initial fused positioning information; According to the initial fused positioning information, a target image region in the image to be processed is identified.

2. The image processing method according to claim 1, characterized in that: The step of determining the candidate positioning weight of each image region according to the region feature information and the preset positioning feature information includes: Calculating at least two initial feature information of the image to be processed according to the regional feature information and the preset positioning feature information; The initial feature information is fused to obtain a candidate positioning weight for each image region.

3. The image processing method according to claim 1, characterized in that: The determining, according to the region feature information, region association information between the image regions comprises: Calculating candidate association parameters between the image regions according to the region feature information and preset positioning feature information; extracting an associated parameter from the candidate associated parameters according to the regional feature information; The region association information between the image regions is generated according to the association parameters.

4. The image processing method according to claim 1, characterized in that: The fusing the image positioning information and the region association information to obtain initial fused positioning information includes: Performing weighted processing on the regional association information according to the image positioning information to obtain weighted positioning information; The initial fused positioning information is determined according to the weighted positioning information.

5. The image processing method according to claim 1, characterized in that: The initial fused positioning information includes a plurality of initial fused positioning information; and identifying the target image area in the image to be processed according to the initial fused positioning information includes: Fusing the plurality of initial fused positioning information to obtain target fused positioning information; According to the target fusion positioning information, a target image area in the image to be processed is identified.

6. The image processing method according to claim 5, characterized in that: The step of identifying the target image area in the image to be processed according to the target fused positioning information includes: Determine the target positioning image corresponding to the image to be processed according to the target fusion positioning information; According to the target positioning image, a target image region in the image to be processed is identified.

7. The image processing method according to claim 1, characterized in that: The regional feature information is feature information obtained by extracting features using a trained image processing model; before extracting features from the image region, the method further includes: Acquire an image sample set, wherein the image sample set includes at least one image sample annotated with a classification label; Encode the image samples using the image processing model to be trained to obtain feature information of the encoded samples; According to the encoded sample feature information, the type of the image sample is predicted to obtain a predicted classification category of the image sample; According to the predicted classification category and the classification label, the model parameters of the image processing model to be trained are converged to obtain a trained image processing model.

8. An image processing device, characterized in that: include: A first acquisition unit, used for acquiring an image to be processed, and cutting the image to be processed to obtain at least two image regions; An extraction unit, used for performing feature extraction on the image region to obtain region feature information of the image region, wherein the region feature is semantic feature information of the image region; The second acquisition unit is used to acquire the image positioning information of the image area according to the area feature information and preset positioning feature information, wherein the preset positioning feature information is feature information preset for positioning the target object in the image to be processed, and the image positioning information is information for positioning the target object in the image to be processed by each image area, including: Determining a candidate positioning weight for each image region according to the region feature information and preset positioning feature information; According to the preset positioning feature information, extracting the positioning weight of each image area from the candidate positioning weights, the positioning weight reflecting the contribution of the corresponding image area to the positioning of the target object in the image to be processed; Using the positioning weight of each image region as image positioning information of each image region; A determining unit, configured to determine, based on the region feature information, region association information between the image regions, wherein the region association information is information on association between the image regions of the image to be processed; A fusion unit, used for fusing the image positioning information and the region association information to obtain initial fused positioning information; The identification unit is used to identify the target image area in the image to be processed according to the initial fused positioning information.

9. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the image processing method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is suitable for being loaded by a processor to execute the image processing method according to any one of claims 1 to 7.

Images