An image recognition method, apparatus, device and medium

Abstract

The present disclosure provides an image recognition method, device, equipment and medium, the method comprises: obtaining an image to be identified and a target task category corresponding to the image to be identified; based on the target task category, determining the target image feature corresponding to the image to be identified through the image recognition model; determining the target image region corresponding to the image to be identified according to the target image feature; wherein the image recognition model can determine different image regions corresponding to the image to be identified based on different task categories. By using the method, since the image recognition model has strong generalization for images of different task categories, image recognition using the image recognition model can improve the image recognition effect, thereby improving the monitoring effect of target monitoring based on the identified image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image recognition method, apparatus, device and medium. Background Technology

[0002] Construction sites are complex and dangerous, so they have high requirements for safety and environmental protection. Regulatory authorities need to monitor construction site facilities to ensure construction safety and avoid environmental problems.

[0003] Because the construction site environment is relatively closed, it is difficult to obtain monitoring data. However, the construction scene is quite complex, and the limited monitoring data that can be obtained is insufficient to achieve good monitoring results.

[0004] Therefore, how to improve the monitoring effect using a small amount of monitoring data has become an urgent problem to be solved. Summary of the Invention

[0005] This disclosure provides an image recognition method, apparatus, device, and medium to at least solve the above-mentioned technical problems existing in the prior art.

[0006] According to a first aspect of this disclosure, an image recognition method is provided, the method comprising:

[0007] Obtain the image to be identified and the target task category corresponding to the image to be identified; wherein, the target task category is one of multiple task categories corresponding to the image to be identified;

[0008] Based on the target task category, the target image features corresponding to the image to be recognized are determined by an image recognition model;

[0009] The target image region in the image to be identified is determined based on the target image features; wherein, the image recognition model can determine different image regions in the image to be identified based on different task categories.

[0010] In one possible implementation, determining the target image features corresponding to the image to be recognized based on the target task category using an image recognition model includes:

[0011] The image to be identified and the target task category are input into the image recognition model;

[0012] Extract the first image features of the image to be identified, and, based on the preset correspondence between task features and task categories, determine the target task features corresponding to the target task category from the task features associated with the image recognition model;

[0013] The target task features and the first image features are fused to obtain the target image features corresponding to the image to be identified.

[0014] In one embodiment, the image recognition model is obtained by training a neural network to be trained based on multiple sample images, each labeled region of each sample image, and the task category to which each labeled region belongs. The task category to which each labeled region belongs includes at least two task categories.

[0015] In one possible implementation, the training method of the image recognition model includes:

[0016] The sample images of the sample image group and the task categories to which the target identification regions of the sample images belong are input into the neural network to be trained. The target identification regions are one of the identification regions of multiple different task categories in the sample images. The sample image group includes multiple sample images, and the task categories to which each identification region of the multiple sample images belongs include at least two task categories.

[0017] Extract the second image features of the sample image, and determine the target sample task features corresponding to the task category to which the target identification region belongs, based on the preset correspondence between sample task features and task categories;

[0018] The target sample task features and the second image features are fused to obtain the fused features corresponding to the sample image, and the predicted image region in the sample image is determined based on the fused features;

[0019] Based on the target identification region in the sample image and the predicted image region, the loss function of the neural network to be trained is determined;

[0020] If the loss function is determined to be convergent, the image recognition model is obtained based on the neural network to be trained.

[0021] If it is determined that the loss function has not converged, the parameters of the neural network to be trained are adjusted, new sample images are obtained from the sample image group, and the process returns to the step of inputting the sample images of the sample image group and the task category to which the target identification region of the sample image belongs into the neural network to be trained.

[0022] In one possible implementation, before determining the loss function of the neural network to be trained based on the target identification region in the sample image and the predicted image region, the method further includes:

[0023] For each sample image, the image region where each target is located in the sample image is determined as a label region, and the type of the target corresponding to each label region is determined as the task type to which the label region belongs.

[0024] In one possible implementation, the target identification region in the sample image is the foreground portion of the sample image, and the other identification regions in the sample image are the background portions of the sample image.

[0025] In one possible implementation, before determining the loss function of the neural network to be trained based on the target identification region in the sample image and the predicted image region, the method further includes:

[0026] The fusion feature is determined as the sample task feature corresponding to the task category to which the target identifier region belongs.

[0027] According to a second aspect of this disclosure, an image recognition apparatus is provided, the apparatus comprising:

[0028] An image acquisition module is used to acquire an image to be identified and a target task category corresponding to the image to be identified; wherein, the target task category is one of multiple task categories corresponding to the image to be identified;

[0029] The image feature determination module is used to determine the target image features corresponding to the image to be identified based on the target task category and through an image recognition model.

[0030] An image recognition module is used to determine the corresponding target image region in the image to be recognized based on the features of the target image; wherein, the image recognition model can determine different image regions in the image to be recognized based on different task categories.

[0031] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0032] At least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0035] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0036] The image recognition method, apparatus, device, and medium disclosed herein acquire an image to be recognized and the target task category corresponding to the image to be recognized. Based on the target task category, the target image features corresponding to the image to be recognized are determined by an image recognition model. The target image region in the image to be recognized is determined based on the target image features. Since the image recognition model has strong generalization for images of different task categories, using the image recognition model for image recognition can improve the image recognition effect, thereby improving the monitoring effect of target monitoring based on the recognized image.

[0037] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0038] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0039] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0040] Figure 1 This illustration shows a schematic diagram of an implementation flow of the image recognition method provided in an embodiment of the present disclosure;

[0041] Figure 2 An image to be identified is shown in an embodiment of this disclosure;

[0042] Figure 3 A schematic diagram illustrating the training process of the image recognition model provided in this embodiment of the present disclosure is shown.

[0043] Figure 4 A schematic diagram of a second image feature provided in an embodiment of this disclosure is shown;

[0044] Figure 5 A schematic diagram of a fusion feature provided in an embodiment of this disclosure is shown;

[0045] Figure 6 A schematic diagram of feature extraction provided in an embodiment of this disclosure is shown;

[0046] Figure 7 A schematic diagram of the image feature determination process provided in an embodiment of this disclosure is shown;

[0047] Figure 8 This illustration shows an image recognition schematic provided by an embodiment of the present disclosure;

[0048] Figure 9A schematic diagram of an image recognition device provided in an embodiment of this disclosure is shown;

[0049] Figure 10 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0050] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0051] Because construction sites are relatively enclosed environments and have complex construction scenarios, monitoring data is difficult to obtain, and the limited amount of data that can be obtained is insufficient for effective monitoring. Therefore, to improve monitoring effectiveness using limited monitoring data, this disclosure provides an image recognition method, apparatus, device, and medium. The method provided in this disclosure can be applied to electronic devices capable of image data processing.

[0052] The technical solutions of the embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0053] Figure 1 This illustration shows a schematic diagram of an implementation flow of the image recognition method provided in an embodiment of the present disclosure, such as... Figure 1 As shown, the method includes:

[0054] S101, Obtain the image to be identified and the target task category corresponding to the image to be identified.

[0055] The target task category is one of multiple task categories corresponding to the image to be identified.

[0056] In this disclosure, the image to be identified is an image collected from a specified monitoring scene, such as a construction site or a traffic section. If the specified monitoring scene is a construction site, then images collected from the construction site can be used as the image to be identified.

[0057] If the specified monitoring scenario is a construction site, the task categories include, but are not limited to: personnel, construction vehicles, construction materials (Category A), and construction materials (Category B). If the specified monitoring scenario is a traffic section, the task categories include, but are not limited to: personnel, vehicles (Category A), and vehicles (Category B).

[0058] The image to be identified may include one or more targets such as people, vehicles and / or construction materials. Each target corresponds to a task category, and the task category corresponding to each target in the image to be identified is the task category corresponding to the image to be identified. Figure 2 An image to be identified provided in an embodiment of this disclosure is shown, such as... Figure 2 As shown, the images 200 collected from the construction site to be identified include dump trucks 201, construction workers 202, tarpaulins 203, and bare soil 204.

[0059] In this disclosure, the target to be monitored can be specified, and the task category to which the target belongs can be determined as the target task category. For example, if a person in an image needs to be monitored, the person category from among multiple task categories corresponding to the image to be identified can be determined as the target task category. For example, for... Figure 2 If the dump truck in the image to be identified is specified in the image 200 to be identified, then the dump truck category among the multiple task categories corresponding to the image 200 to be identified can be determined as the target task category.

[0060] S102, Based on the target task category, determine the target image features corresponding to the image to be recognized through an image recognition model.

[0061] S103 determines the corresponding target image region in the image to be identified based on the target image features.

[0062] The image recognition model can determine different image regions in the image to be recognized based on different task categories.

[0063] The image recognition method disclosed herein obtains the image to be recognized and the target task category corresponding to the image to be recognized. Based on the target task category, the target image features corresponding to the image to be recognized are determined by the image recognition model. The target image region in the image to be recognized is determined according to the target image features. Since the image recognition model has strong generalization for images of different task categories, using the image recognition model for image recognition can improve the image recognition effect, thereby improving the monitoring effect of target monitoring based on the recognized image.

[0064] In this disclosure, the image recognition model is obtained by training a neural network to be trained based on multiple sample images, each labeled region of each sample image, and the task category to which each labeled region belongs. The task category to which each labeled region belongs includes at least two task categories.

[0065] Among them, sample images refer to images collected from a specified monitoring environment. For example, if the specified monitoring scenario is a construction site, then images collected from the construction site can be used as sample images to train the neural network to be trained.

[0066] However, due to the relatively enclosed environment and complex construction scenarios at construction sites, the number of sample images that can be collected is usually small. In order to enhance the generalization ability of the image recognition model trained using sample images, this disclosure can enrich the training data by labeling the collected sample images with the marked regions and the task categories to which the marked regions belong, thereby improving the generalization ability of the trained image recognition model.

[0067] In this disclosure, for each sample image, the image region where each target in the sample image is located is determined as a label region, and the type of the target corresponding to each label region is determined as the task type to which the label region belongs.

[0068] Specifically, for each sample image, the image regions containing each target within the image can be labeled, and these regions are designated as the target's identification regions. For example, if sample image M1 includes object A and object B, where object A belongs to the construction vehicle category and object B belongs to the construction material category B, then the image region containing object A in sample image M1 can be labeled as identification region A, and the image region containing object B in sample image M1 can be labeled as identification region B. Furthermore, the task category corresponding to identification region A is the construction vehicle category, and the task category corresponding to identification region B is the construction material category B.

[0069] For a sample image with labeled regions, one labeled region in the sample image can be designated as the target labeled region, and the target labeled region can be defined as the foreground portion of the sample image. The other labeled regions in the sample image can be designated as the background portion, resulting in new sample images. After obtaining multiple new sample images, they can be divided into sample image groups, each containing a predetermined number of new sample images for training the neural network. The foreground portions of the predetermined number of new sample images in each sample image group belong to at least two task categories. The predetermined number can be set to 10 or 20, etc. In this disclosure, by dividing a sample image into foreground and background portions, a single sample image can be expanded into multiple new sample images with different foreground regions, enriching the training data.

[0070] In one possible implementation, Figure 3 This illustration shows a schematic diagram of the training process of the image recognition model provided in an embodiment of this disclosure, such as... Figure 3 As shown, the training methods for the image recognition model include:

[0071] S301, input the sample images of the sample image group and the task category to which the target identification region of the sample images belongs into the neural network to be trained.

[0072] The target identification region is one of the identification regions of multiple different task categories in the sample image. The sample image group includes multiple sample images, and the task categories to which each identification region of the multiple sample images belongs include at least two task categories.

[0073] In the sample image group, the target identification area of ​​each sample image is labeled as the foreground part of the image, and the other identification areas of the sample image are labeled as the background part of the image.

[0074] S302, extract the second image features of the sample image, and determine the target sample task features corresponding to the task category to which the target identification region belongs based on the preset correspondence between sample task features and task categories.

[0075] In this step, color features, texture features, shape features, and / or spatial relationship features of the sample image can be extracted as the second image features of the sample image.

[0076] In this disclosure, for each task category, the second image feature of the first sample image whose task category the target identification region extracted by the neural network to be trained belongs to can be used as the initial preset sample task feature corresponding to that task category. For example, if the task category is "person", the target identification region of sample image X is the region where the person is located, and the second image feature corresponding to sample image X is the image feature of the first "person" category extracted by the neural network to be trained, then the second image feature corresponding to sample image X can be used as the initial preset sample task feature corresponding to the "person" category.

[0077] If there is no preset sample task feature corresponding to the task category to which the target identification region belongs in the current iteration, then the vector with the same dimensions as the second image feature of the sample image and with each dimension value of 0 can be determined as the target sample task feature.

[0078] S303, the target sample task features and the second image features are fused to obtain the fused features corresponding to the sample image, and the predicted image region in the sample image is determined based on the fused features.

[0079] For example, Figure 4 This illustration shows a schematic diagram of a second image feature provided in an embodiment of the present disclosure. Figure 4The second image feature shown includes three vector dimensions: dimension 1, dimension 2, and dimension 3. Specifically, it consists of vectors 410, 420, and 430 of dimensions 1 and 2. If the target sample task feature is a zero vector with only two dimensions (dimension 1 and dimension 2) and the same number of elements as vector 410, then the target sample task feature and the second image feature can be concatenated along dimension 3 to obtain the following result. Figure 5 The stitching features shown are used as the fusion features corresponding to the sample images, such as... Figure 5 As shown, the all-zero vector 510 represents the target sample task features.

[0080] After obtaining the fused features, the neural network to be trained can decode the fused features to decode the regions corresponding to each target in the sample image as the predicted image regions.

[0081] Figure 6 A schematic diagram of feature extraction provided in an embodiment of this disclosure is shown, such as... Figure 6 As shown, the sample image group includes sample image 1, sample image 2, and sample image 3. In sample image 1, the target area where target 1 is located is marked as the foreground part of the image, and the other areas are the background part of the image. The task type of target 1 is task type A. In sample image 2, the target area where target 2 is located is marked as the foreground part of the image, and the other areas are the background part of the image. The task type of target 2 is task type B. In sample image 3, the target area where target 3 is located is marked as the foreground part of the image, and the other areas are the background part of the image. The task type of target 3 is task type C. The task type of the target area of ​​each sample image is input into the neural network to be trained, and the second image features of each sample image are extracted. Then, the target sample task features corresponding to the task type of the target area of ​​the sample image are fused with the second image features of the sample image to obtain the fused features corresponding to the sample image. For example, the fused features of sample image 1 are feature vector 1, the fused features of sample image 2 are feature vector 2, and the fused features of sample image 3 are feature vector 3.

[0082] Furthermore, the sample task features corresponding to each task category can be updated based on the fusion features of each sample image. Specifically, the fusion features can be determined as the sample task features corresponding to the task category to which the target identification region belongs. For example, feature vector 1 can be updated to the sample task features corresponding to task category A, feature vector 2 can be updated to the sample task features corresponding to task category B, and feature vector 3 can be updated to the sample task features corresponding to task category C. When the model training ends, the task-specific vector composed of the sample task features corresponding to each task category in the image recognition model can be used to identify the image to be recognized.

[0083] S304, Based on the target identification region in the sample image and the predicted image region, determine the loss function of the neural network to be trained.

[0084] In this disclosure, a logarithmic loss function or a cross-entropy loss function can be calculated between the target identification region in the sample image and the predicted image region, and used as the loss function of the neural network to be trained.

[0085] S305, if it is determined that the loss function has converged, the image recognition model is obtained based on the neural network to be trained.

[0086] S306, if it is determined that the loss function has not converged, adjust the parameters of the neural network to be trained, obtain new sample images in the sample image group, and return to execute the step of inputting the sample images of the sample image group and the task category to which the target identification region of the sample image belongs into the neural network to be trained.

[0087] Specifically, the parameters in the neural network being trained that affect the loss function can be adjusted.

[0088] In one possible implementation, Figure 7 A schematic diagram of the image feature determination process provided in an embodiment of this disclosure is shown, such as... Figure 7 As shown, determining the target image features corresponding to the image to be recognized based on the target task category using an image recognition model includes:

[0089] S701, input the image to be recognized and the target task category into the image recognition model.

[0090] S702, extract the first image features of the image to be recognized, and determine the target task features corresponding to the target task category from the task features associated with the image recognition model according to the preset correspondence between task features and task categories.

[0091] S703, the target task features and the first image features are fused to obtain the target image features corresponding to the image to be identified.

[0092] For example, Figure 8 This illustration shows an image recognition schematic provided by an embodiment of the present disclosure, such as... Figure 8As shown, the image recognition model includes an input layer, a backbone network layer, a decoder, and an output layer. Furthermore, the model includes task-specific vectors, which contain preset task features corresponding to each task category. After inputting the image to be recognized (A) and the target task category into the input layer, the backbone network of the image recognition model can extract the first image feature of image A. Based on the task-specific vectors included in the image recognition model, the preset task features corresponding to the target task category can be determined. For example, the task-specific vector corresponding to task category 1 is feature vector 1, the task-specific vector corresponding to task category 2 is feature vector 2, and the task-specific vector corresponding to task category 3 is feature vector 3. If the target task category is task category 1, then the target task feature corresponding to the target task category can be determined as feature vector 1. Feature vector 1 is fused with the first image feature of image A to obtain a fused feature. Then, the decoder is used to decode the fused feature to determine the recognition result. The recognition result includes the image regions corresponding to each target in image A, and the output layer can output the recognition result. For example, if the targets in the image A to be identified include construction workers W, dump truck Y, and tarpaulin Z on a construction site, the identification result can display the target image region 1 corresponding to construction worker W, the target image region 2 corresponding to dump truck Y, and the target image region 3 corresponding to tarpaulin Z in the image A to be identified.

[0093] The image recognition method disclosed herein trains a highly generalizable image recognition model using sample images from different task categories. When performing image recognition, it can combine the target task features corresponding to the target task category to which the image belongs to identify the target region in the image to be identified, thereby improving the image recognition effect and thus improving the monitoring effect of target monitoring based on the identified image.

[0094] Based on the same inventive concept, and according to the image recognition method provided in the above embodiments of this disclosure, another embodiment of this disclosure also provides an image recognition device, the structural schematic diagram of which is shown below. Figure 9 As shown, it specifically includes:

[0095] Image acquisition module 901 is used to acquire an image to be identified and a target task category corresponding to the image to be identified; wherein, the target task category is one of multiple task categories corresponding to the image to be identified;

[0096] The image feature determination module 902 is used to determine the target image features corresponding to the image to be identified based on the target task category and through an image recognition model.

[0097] The image recognition module 903 is used to determine the corresponding target image region in the image to be recognized based on the target image features; wherein, the image recognition model can determine different image regions in the image to be recognized based on different task categories.

[0098] The image recognition device disclosed herein acquires the image to be recognized and the target task category corresponding to the image to be recognized. Based on the target task category, the image recognition model determines the target image features corresponding to the image to be recognized. Based on the target image features, the corresponding target image region in the image to be recognized is determined. Since the image recognition model has strong generalization for images of different task categories, using the image recognition model for image recognition can improve the image recognition effect, thereby improving the monitoring effect of target monitoring based on the recognized image.

[0099] In one embodiment, the image feature determination module 902 is specifically used to input the image to be identified and the target task category into the image recognition model; extract the first image feature of the image to be identified; and, according to the preset correspondence between task features and task categories, determine the target task feature corresponding to the target task category from the task features associated with the image recognition model; and fuse the target task feature and the first image feature to obtain the target image feature corresponding to the image to be identified.

[0100] In one embodiment, the image recognition model is obtained by training a neural network to be trained based on multiple sample images, each labeled region of each sample image, and the task category to which each labeled region belongs. The task category to which each labeled region belongs includes at least two task categories.

[0101] In one possible embodiment, the device further includes:

[0102] The model training module (not shown in the figure) is used to input sample images from a sample image group and the task categories to which the target identification regions of the sample images belong to the neural network to be trained. The target identification region is one of multiple identification regions of different task categories in the sample images. The sample image group includes multiple sample images, and the task categories to which each identification region of the multiple sample images belongs include at least two task categories. The module extracts second image features from the sample images and, based on a preset correspondence between sample task features and task categories, determines the target sample task features corresponding to the task category to which the target identification region belongs. The module then compares the target sample task features with the second image features of the sample images. The two image features are fused to obtain the fused features corresponding to the sample image, and the predicted image region in the sample image is determined based on the fused features. The loss function of the neural network to be trained is determined based on the target identification region in the sample image and the predicted image region. If the loss function is determined to be converged, the image recognition model is obtained based on the neural network to be trained. If the loss function is determined to be unconverged, the parameters of the neural network to be trained are adjusted, new sample images in the sample image group are obtained, and the process returns to the step of inputting the sample images in the sample image group and the task category to which the target identification region of the sample image belongs into the neural network to be trained.

[0103] In one embodiment, the model training module is further configured to, for each sample image, determine the image region where each target in the sample image is located as a label region, and determine the type of the target corresponding to each label region as the task type to which the label region belongs.

[0104] In one possible implementation, the target identification region in the sample image is the foreground portion of the sample image, and the other identification regions in the sample image are the background portions of the sample image.

[0105] In one embodiment, the model training module is further configured to determine the fused features as sample task features corresponding to the task category to which the target identifier region belongs.

[0106] The image recognition device of this disclosure, by training a highly generalized image recognition model using sample images of different task categories, can identify target regions in the image by combining the target task features corresponding to the target task category to which the image belongs, thereby improving the image recognition effect and thus improving the monitoring effect of target monitoring based on the recognized image.

[0107] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0108] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0110] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0111] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as image recognition methods. For example, in some embodiments, the image recognition method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the image recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform image recognition methods by any other suitable means (e.g., by means of firmware).

[0112] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0116] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0117] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0118] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0120] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image recognition method, the method comprising: Obtain the image to be identified and the target task category corresponding to the image to be identified; wherein, the target task category is one of multiple task categories corresponding to the image to be identified; Input the image to be identified and the target task category into the image recognition model; Extract the first image features of the image to be identified, and, based on the preset correspondence between task features and task categories, determine the target task features corresponding to the target task category from the task features associated with the image recognition model; The target task features and the first image features are fused to obtain the target image features corresponding to the image to be identified; The target image region in the image to be identified is determined based on the target image features; wherein, the image recognition model can determine different image regions in the image to be identified based on different task categories.

2. The method according to claim 1, wherein the image recognition model is obtained by training a neural network to be trained based on multiple sample images, each identifier region of each sample image and the task category to which each identifier region belongs, and the task category to which each identifier region belongs includes at least two task categories.

3. The method according to claim 2, wherein the training method of the image recognition model includes: The sample images of the sample image group and the task categories to which the target identification regions of the sample images belong are input into the neural network to be trained. The target identification regions are one of the identification regions of multiple different task categories in the sample images. The sample image group includes multiple sample images, and the task categories to which each identification region of the multiple sample images belongs include at least two task categories. Extract the second image features of the sample image, and determine the target sample task features corresponding to the task category to which the target identification region belongs, based on the preset correspondence between sample task features and task categories; The target sample task features and the second image features are fused to obtain the fused features corresponding to the sample image, and the predicted image region in the sample image is determined based on the fused features; Based on the target identification region in the sample image and the predicted image region, the loss function of the neural network to be trained is determined; If the loss function is determined to be convergent, the image recognition model is obtained based on the neural network to be trained. If it is determined that the loss function has not converged, the parameters of the neural network to be trained are adjusted, new sample images are obtained from the sample image group, and the process returns to the step of inputting the sample images of the sample image group and the task category to which the target identification region of the sample image belongs into the neural network to be trained.

4. The method according to claim 3, before determining the loss function of the neural network to be trained based on the target identification region in the sample image and the predicted image region, the method further includes: For each sample image, the image region where each target is located in the sample image is determined as a label region, and the type of the target corresponding to each label region is determined as the task type to which the label region belongs.

5. The method according to claim 3, wherein the target identification region in the sample image is the foreground portion of the sample image, and the other identification regions in the sample image are the background portions of the sample image.

6. The method according to claim 3, before determining the loss function of the neural network to be trained based on the target identification region in the sample image and the predicted image region, the method further includes: The fusion feature is determined as the sample task feature corresponding to the task category to which the target identifier region belongs.

7. An image recognition device, the device comprising: An image acquisition module is used to acquire an image to be identified and a target task category corresponding to the image to be identified; wherein, the target task category is one of multiple task categories corresponding to the image to be identified; An image feature determination module is used to input the image to be identified and the target task category into an image recognition model; extract the first image feature of the image to be identified; and, based on a preset correspondence between task features and task categories, determine the target task feature corresponding to the target task category from the task features associated with the image recognition model; and fuse the target task feature and the first image feature to obtain the target image feature corresponding to the image to be identified. An image recognition module is used to determine the corresponding target image region in the image to be recognized based on the features of the target image; wherein, the image recognition model can determine different image regions in the image to be recognized based on different task categories.

8. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-6.

Images