Image recognition method and apparatus, and computing device and storage medium

Abstract

Provided in the present disclosure are an image recognition method and apparatus, and a computing device and a storage medium. The image recognition method comprises: acquiring a target recognition task, wherein the target recognition task comprises at least one task type and an image to be subjected to recognition; on the basis of the at least one task type, determining, from among a plurality of pre-trained task components, at least one target task component corresponding to each task type; determining a target recognition model on the basis of a basic component and the at least one target task component corresponding to each task type, wherein the basic component and the plurality of pre-trained task components are obtained by means of performing training on the basis of task samples of different recognition tasks, and different recognition tasks share the same basic component; and inputting said image into the target recognition model, so as to determine, from said image, a recognition result of a target object corresponding to each task type.

Description

Image recognition method, device, computing device and storage medium

[0001] This application claims priority to Chinese patent application No. 202310779095.0 filed on June 28, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of robot control technology, and in particular to an image recognition method, an image recognition device, a computing device, and a computer-readable storage medium. Background Art

[0003] At present, intelligent driving is booming, and unmanned vehicles are being used more and more widely in unmanned warehousing environments as well as in human-machine mixed factory and workshop environments.

[0004] In autonomous vehicle scenarios, autonomous vehicles typically follow a set path to perform their mission. Path recognition primarily relies on understanding the surrounding environment to determine the direction of travel. For example, autonomous vehicles need to identify road markings to determine their path and identify obstacles and pedestrians for avoidance.

[0005] To ensure safe driving, autonomous vehicles must perform multiple recognition operations on captured environmental images. For example, they must identify road markings, obstacles, and pedestrians in the captured environmental images to ensure safe driving. However, this process consumes a significant amount of resources. If each object to be identified in the environmental image is recognized once, the resource consumption increases linearly. Therefore, a method is urgently needed to reduce the resource consumption of identifying objects in the environment.

[0006] Summary of the Invention

[0007] Embodiments of the present disclosure provide an image recognition method, apparatus, computing device, and storage medium.

[0008] According to a first aspect of an embodiment of the present disclosure, there is provided an image recognition method, comprising: obtaining a target recognition task; wherein the target recognition task includes at least one task type and an image to be recognized; based on the at least one task type, determining at least one target task component corresponding to each task type from a plurality of pre-trained task components; determining a target recognition model based on a basic component and at least one target task component corresponding to each task type; wherein the basic component and the plurality of pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component; inputting the image to be recognized into the target recognition model, and determining the recognition results of the target objects corresponding to each task type in the image to be recognized.

[0009] According to a second aspect of an embodiment of the present disclosure, an image recognition device is provided, comprising: an acquisition module configured to acquire a target recognition task; wherein the target recognition task includes at least one task type and an image to be recognized; a determination module configured to determine, based on the task type, at least one target task component corresponding to each task type in multiple pre-trained task components; an assembly module configured to determine a target recognition model based on a basic component and at least one target task component corresponding to each task type; wherein the basic component and multiple pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component; an input module configured to input the image to be recognized into the target recognition model, and determine the recognition result of the target object corresponding to each task type in the image to be recognized.

[0010] According to a third aspect of an embodiment of the present disclosure, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor implements the steps of the image recognition method described in the first aspect when executing the computer-executable instructions.

[0011] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the image recognition method described in the first aspect are implemented.

[0012] According to a fifth aspect of an embodiment of the present disclosure, a computer program is provided. When the computer program is executed by a processor, the steps of the image recognition method described in the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG1 is a schematic diagram of an image recognition method provided by an embodiment of the present disclosure;

[0014] FIG2 is a schematic diagram of a target recognition model structure provided by an embodiment of the present disclosure;

[0015] FIG3A is a schematic diagram of another image recognition method provided by an embodiment of the present disclosure;

[0016] FIG3B is a schematic diagram of a recognition model training process provided by an embodiment of the present disclosure;

[0017] FIG3C is a schematic diagram of an image recognition process provided by an embodiment of the present disclosure;

[0018] FIG3D is a schematic diagram of a recognition model update training according to an embodiment of the present disclosure;

[0019] FIG4 is a schematic diagram of an image recognition device provided by an embodiment of the present disclosure;

[0020] FIG5 is a structural block diagram of a computing device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following description sets forth many specific details to facilitate a full understanding of the present disclosure. However, the present disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present disclosure. Therefore, the present disclosure is not limited to the specific implementations disclosed below.

[0022] The terms used in one or more embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present disclosure. The singular forms "a", "the", and "the" used in one or more embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms "first," "second," and the like may be used to describe various information in one or more embodiments of the present disclosure, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, "first" may also be referred to as "second," and similarly, "second" may also be referred to as "first," without departing from the scope of one or more embodiments of the present disclosure.

[0024] First, the terms involved in one or more embodiments of the present disclosure are explained.

[0025] Autonomous Mobile Robot (AMR): In a broad sense, it refers to a mobile robot with strong autonomy.

[0026] At present, intelligent driving is booming, and unmanned vehicles are being used more and more widely in unmanned warehousing environments as well as in human-machine mixed factory and workshop environments.

[0027] In autonomous vehicle scenarios, autonomous vehicles typically follow a set path to perform their mission. Path recognition primarily relies on understanding the surrounding environment to determine the direction of travel. For example, autonomous vehicles need to identify road markings to determine their path and identify obstacles and pedestrians for avoidance.

[0028] To ensure safe driving, autonomous vehicles must perform multiple recognition operations on captured environmental images. For example, during driving, they must identify road markings, obstacles, and pedestrians in the captured environmental images to ensure safe driving. However, recognizing environmental images consumes a significant amount of resources. If each object to be identified in the environmental image is recognized once, the resource consumption increases linearly. Furthermore, each recognition task cannot utilize information from other tasks to assist in detection, resulting in certain limitations in image recognition.

[0029] The present disclosure provides an image recognition method, which comprises obtaining a target recognition task, the target recognition task including a task type and an image to be recognized; selecting a target task component from a plurality of pre-trained task components based on the task type; assembling a base component and a target task component to obtain a target recognition model, wherein the base component and the plurality of pre-trained task components are trained based on task samples of different recognition tasks, and the different recognition tasks share the same base component; inputting the image to be recognized into the target recognition model to obtain a recognition result of the target object in the image to be recognized. The present disclosure provides an image recognition method, which, upon obtaining a target recognition task, can determine a target task component from a plurality of pre-trained task components based on the task type in the target recognition task, assemble the target task component with the base component shared by the different recognition tasks to obtain a target recognition model, and recognize the image to be recognized in the target recognition task using the target recognition model to obtain a recognition result. In other words, the present disclosure can use the base component shared by different recognition tasks so that, during the task recognition process, features output by the base component can be used for corresponding processing in the target task component, thereby avoiding the repeated consumption of resources used by the corresponding processing functions of the base component when recognizing different recognition tasks and reducing the resource consumption when recognizing the recognition task.

[0030] FIG1 is a schematic diagram of an image recognition method provided by an embodiment of the present disclosure, wherein the image recognition method includes the following steps:

[0031] Step 102: Obtain target recognition task.

[0032] The target recognition task includes at least one task type and an image to be recognized.

[0033] In some examples, when there is a need to identify a task, an object recognition task including a task type and an image to be identified is obtained. The object recognition task refers to a task to be identified, and the object recognition task may include at least one task type and an image to be identified.

[0034] For example, the task type refers to the recognition type corresponding to the task to be recognized. For example, if the task to be recognized is a QR code recognition task, the task type is QR code recognition. The image to be recognized refers to the image to be recognized. The image to be recognized can include task objects corresponding to multiple tasks to be recognized. For example, in a warehouse environment, the image to be recognized can be an environmental image captured by a robot. The shelves, pedestrians, bins, pallets, markings, obstacles, etc. included in the environmental image can all be target objects.

[0035] In some examples, acquiring a target recognition task may include: acquiring the target recognition task upon receiving a recognition task instruction. For example, upon receiving a task recognition instruction from a control center, the robot may acquire the captured image to be recognized and the task type contained in the task recognition instruction based on the task recognition instruction. Alternatively, the robot may acquire the target recognition task at preset intervals based on a pre-set program to perform recognition tasks.

[0036] After the target recognition task is acquired, a target task component is selected based on the task type included in the target recognition task, and subsequent image recognition processing is performed based on the target task component and the image to be recognized to obtain a recognition result of the target object in the image to be recognized.

[0037] Step 104: Based on at least one task type, determine at least one target task component corresponding to each task type from a plurality of pre-trained task components.

[0038] In some examples, a task component refers to a component that processes image features corresponding to a recognition task to obtain an image recognition result. A task component (also referred to as a head component) may include an attention unit and a task head unit. A target task component refers to a component that processes features corresponding to an image to be recognized based on the task type. The target task component corresponds to the task type and includes a target attention unit and a target task head unit.

[0039] Exemplarily, determining an implementation method of a target task component among multiple pre-trained task components based on the task type may include: searching a component library storing multiple pre-trained task components based on the task type, and determining the target task component based on the search result.

[0040] In some examples, based on the task type, when a task component corresponding to each task type is found in the component library, the found task component is determined as the target task component.

[0041] In other examples, based on the task type, if the task component corresponding to the task type cannot be found in the component library, that is, if the target task component does not exist, a sample set corresponding to the target recognition task can be obtained, and the initial task component corresponding to the target recognition model can be added to the pre-trained recognition model. The parameters of the initial task component can be adjusted using the sample set corresponding to the target recognition task to obtain the target task component corresponding to the target recognition task; the adjusted target task component is determined as the target task component, and the basic component (also called the backbone component) and the target task component are subsequently assembled to obtain the target recognition model. The basic component and the target task component are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component; the image to be recognized is input into the target recognition model to obtain the recognition result of the target object in the image to be recognized.

[0042] In some examples, the number of task types in a target recognition task can be one or more. If there are multiple task types, there are also multiple target recognition tasks. The multiple target recognition tasks can correspond one-to-one with the multiple task types; alternatively, multiple target recognition tasks can correspond to one task type, and different task types can correspond to different task components.

[0043] For example, when there is only one task type, the task component corresponding to the task type among multiple pre-trained task components can be used as the target task component. When there are multiple task types, the task components corresponding to each task type can be determined from the multiple pre-trained task components, and the task components corresponding to each task type can be determined as the target task component. For example, when there are two task types, the task components corresponding to the two task types can be selected as the target task components.

[0044] In some examples, one task type may correspond to at least one task component, that is, each task type may correspond to one target task component or may correspond to multiple target task components.

[0045] Step 106: Determine a target recognition model based on the basic components and at least one target task component corresponding to each task type.

[0046] Among them, the basic component and multiple pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component.

[0047] Exemplarily, the above step 106 may include: assembling the basic components and each target task component to obtain a target recognition model.

[0048] In some examples, assembling the base component and the target task component may include assembling the base component and the target task according to a preset structure (e.g., a first preset structure). The preset structure includes a single-task component, a dual-task component, and a multi-task component. For example, a single-task component has a base component and a task component connected in series, a dual-task component has two task components connected in parallel, and the resulting parallel component is connected in series with the base component, and a multi-task component has a plurality of task components connected in parallel, and the resulting parallel component is connected in series with the base component.

[0049] In some examples, when there are multiple target task components, assembling the base component and each target task component to obtain a target recognition model includes: assembling the base component and each target task component separately according to a preset structure to obtain the target recognition model. Assembling the base component and each target task component separately according to the preset structure to obtain the target recognition model may include connecting each target task component in parallel and connecting the result of the parallel connection in series with the base component to obtain the target recognition model.

[0050] The image recognition method provided by the embodiment of the present disclosure selects target task components corresponding to each task type from multiple pre-trained task components based on each task type, assembles the basic components and the target task components, and obtains a target recognition model, so that the obtained target recognition model can handle recognition tasks when there are multiple task types, thereby realizing efficient recognition processing of multiple task types.

[0051] In some embodiments, when there are multiple target task components, the above step 106 may include: processing multiple target task components in parallel and / or in series to obtain a target task component set; and connecting the basic component and the target task component set in series to obtain a target recognition model.

[0052] In some embodiments, multiple target task components are processed in parallel and / or in series, including: connecting at least two target task components corresponding to the same task type among the multiple target task components in series to obtain a target task component subset; and connecting multiple target task component subsets corresponding to different task types in parallel to obtain a target task component set.

[0053] In some examples, since a task type can correspond to multiple target task components, when a task type corresponds to multiple target task components, these multiple target task components can be connected in series to obtain a target task component subset; if there are multiple different task component sets, these multiple different task component sets can be connected in parallel to obtain a target task component set.

[0054] For example, at least two target task components corresponding to the same type are connected in series to obtain a target task component subset, including: identifying the task type corresponding to the target task component, connecting the target task components belonging to the same task type in series to obtain a target task component subset corresponding to each task type.

[0055] In some examples, connecting subsets of target task components corresponding to different task types in parallel to obtain a target task component set includes placing the subsets of target task components corresponding to different task types in corresponding positions according to a preset structure (e.g., a third preset structure) to obtain the target task component set. The preset structure refers to a structure in which each task component is connected in parallel and each task component is connected in series with the base component.

[0056] Figure 2 is a schematic diagram of a target recognition model structure provided by an embodiment of the present disclosure. As shown in Figure 2, the target recognition task includes a QR code recognition task and a pedestrian recognition task, and the QR code recognition task includes QR code recognition subtask 1 and QR code recognition subtask 2.

[0057] Among them, QR code recognition subtask 1 and QR code recognition subtask 2 are of the same task type, that is, QR code recognition subtask 1 and QR code recognition subtask 2 both recognize QR codes, but because QR code recognition subtask 1 and QR code recognition subtask 2 include different images to be recognized, QR code recognition subtask 1 and QR code recognition subtask 2 correspond to different target task components. For example, the image to be recognized included in QR code recognition subtask 1 is the upper half of the target image, the image to be recognized included in QR code recognition subtask 2 is the lower half of the target image, and the image to be recognized included in the pedestrian recognition task is the target image. The target task component corresponding to QR code recognition subtask 1 is QR code recognition task component 1, and the target task component corresponding to QR code recognition subtask 2 is QR code recognition task component 2. Among them, the functions that can be achieved by QR code recognition task component 1 and QR code recognition task component 2 are the same; the target task component corresponding to the pedestrian recognition task is the pedestrian recognition task component.

[0058] As shown in Figure 2, QR code recognition task component 1 and QR code recognition task component 2 can be connected in series to obtain a QR code recognition task component subset corresponding to the QR code recognition task. The pedestrian recognition task component can also be called a pedestrian recognition task component subset. The QR code recognition task component subset and the pedestrian recognition task component subset are connected in parallel to obtain a target task component set. The basic component and the target task component set are then connected in series to generate a target recognition model that performs recognition processing for the target recognition task.

[0059] The image recognition method provided by the embodiment of the present disclosure connects in series at least two target task components corresponding to the same task type to obtain a target task component subset, and connects in parallel the target task component subsets corresponding to different task types to obtain a target task component set, so that the target task component set and the basic component can be subsequently connected in series to obtain a target recognition model. In the application of the target recognition model, multiple target recognition tasks belonging to the same task type and target recognition tasks belonging to different task types can be processed, thereby realizing efficient processing of the target recognition model and efficient application of the target recognition model.

[0060] In some examples, after determining the basic components and the target task component set, the basic components and the target task component set can be connected in series to obtain a target recognition model.

[0061] Exemplarily, connecting the basic components and the target task component set in series to obtain the target recognition model includes: placing the basic components and the target task component set in series according to a preset structure (e.g., a second preset structure) to obtain the target recognition model. The preset structure refers to a structure in which each task component is connected in parallel, and the basic components and the task components are connected in series.

[0062] In some examples, during the application phase of the target recognition model, after obtaining the recognition results output by each target task component, the outputs of the target task components belonging to the same task type can be added together to obtain the recognition results corresponding to each task type.

[0063] In some embodiments, the target recognition model may further include a neck component. When the target recognition model further includes the neck component, step 106 includes: connecting the base component, the neck component, and the target task component set in series to obtain the target recognition model; wherein the neck component is located between the base component and the target task component set.

[0064] In some examples, a base component, a neck component (also called a neck component), and a target task component set can be connected in series to obtain an object recognition model. The neck component is usually located between the base component and the task component to enhance the features output by the base component so that the features extracted by the base component can be better utilized when processing the recognition task.

[0065] The image recognition method provided by the embodiment of the present disclosure connects the target task components in parallel to obtain a target task component set, connects the basic component and the target task component set in series to obtain a target recognition model, and connects the target task components in parallel and connects the target task component set obtained in parallel with the basic component in series to obtain a target recognition model that can identify and process task types corresponding to multiple target task components.

[0066] In some embodiments, the task components may include an attention unit and a task head unit. Step 106 includes: based on at least one task type, determining a target attention unit and a target task head unit corresponding to each task type from a plurality of pre-trained attention units and a plurality of task head units; and determining each target task component based on the target attention unit and the target task head unit.

[0067] An attention unit is a unit that performs attention calculations on features. An attention unit can include a multilayer perceptron (MLP) and an attention layer. A target attention unit is an attention unit corresponding to the target task component. A task head unit is a unit that uses features to predict outcomes. A target task head unit is a task head unit corresponding to the target task component.

[0068] In some examples, based on the task type, a target task type can be searched from a unit library storing pre-trained tasks, and the attention unit and task head unit corresponding to the target task type can be determined as the target attention unit and target task head unit; wherein the unit library records the correspondence between the task type and the attention unit and task head, as well as the parameters of the attention unit and task head unit. Based on the task type, searching for the target task type from the unit library storing pre-trained tasks can include comparing the task type with multiple initial task types in the unit library, and determining the target task type from the multiple initial task types based on the comparison results.

[0069] In other examples, semantic features of the task type can also be extracted based on the task type, and the semantic features can be compared with the semantic features of each attention unit and task head unit, and the target attention unit and target task head unit corresponding to the task type can be determined based on the comparison results.

[0070] The image recognition method provided by the embodiment of the present disclosure, when the target task component includes a target attention unit and a target task head unit, selects the target attention unit and target task head unit corresponding to the task type from the pre-trained attention units and task head units based on the task type, so that the target attention unit and target task head unit based on the selected are subsequently combined with the basic components to obtain image recognition processing for the image to be recognized.

[0071] For example, after determining the target task component corresponding to the target recognition model, it is necessary to assemble the target task component with the basic components shared by different recognition tasks to obtain the target recognition model, so that the target recognition model can perform corresponding processing on the target recognition task and obtain the recognition result corresponding to the target recognition task.

[0072] In some examples, the target recognition model refers to a model that recognizes an image and obtains recognition results of objects in the image. The target recognition model includes at least a basic component and a task component, and may also include a neck component.

[0073] For example, assembling basic components and target task components to obtain a target recognition model includes: based on a preset structure of the recognition model (such as a third preset mechanism), assembling the basic components and the target task components to obtain a target recognition model, wherein the preset structure refers to the recognition model including the basic components and the task components, as well as the arrangement between the basic components and the task components.

[0074] In some examples, the recognition model may also include a neck component, which is used to enhance the features output by the basic component and transmit the enhanced features to the task component; the implementation method of assembling the basic component and the target task component to obtain the target recognition model may be to assemble the basic component, the neck component and the target task component to obtain the target recognition model.

[0075] Step 108: Input the image to be recognized into the target recognition model to determine the recognition results of the target objects corresponding to each task type in the image to be recognized.

[0076] For example, the image to be recognized may include multiple target objects, wherein one task type may correspond to at least one target object, that is, one task type may correspond to some or all of the multiple target objects.

[0077] In some examples, a recognition result refers to a prediction result obtained by applying a target recognition result output to an image to be recognized. For example, the recognition result may include the location information of the target object. For example, based on multiple recognition results, information about the speed of movement of the target object may be obtained.

[0078] Exemplarily, inputting the image to be identified into the target recognition model to obtain the recognition result of the target object in the image to be identified includes: inputting the image to be identified into the target recognition model, and performing corresponding processing by each component in the target recognition model to obtain the recognition result of the target object in the image to be identified.

[0079] In some examples, the target task component in the target recognition model includes a target attention unit and a target task head unit, wherein the target attention unit includes a target attention matrix corresponding to the target recognition task.

[0080] In some embodiments, the above step 108 may include: inputting the image to be identified into the basic components of the target recognition model to extract basic image features; obtaining weighted image features based on the basic image features and the target attention unit; and determining the recognition results of the target objects corresponding to each task type in the image to be identified based on the target task head unit and the weighted image features.

[0081] In some examples, inputting the image to be identified into the basic component of the target recognition model and extracting basic image features may include: inputting the image to be identified into the target recognition model, performing dimensionality conversion on the image to be identified through the basic component, and obtaining basic image features, wherein the dimensionality conversion may be dimensionality increase or dimensionality reduction, which is determined based on actual conditions and is not limited in this disclosure.

[0082] In other examples, the basic component may include multiple network layers, input the image to be identified into the basic component in the target recognition model, and extract basic image features. It may also include: inputting the image to be identified into the target recognition model, and obtaining the basic image features output by each network layer through extraction processing by each network layer in the basic component.

[0083] For example, after extraction and processing of each network layer in the basic component, the basic image features output by each network layer are obtained, including: inputting the image to be identified into the first network layer among multiple network layers, and obtaining the basic image features output by the first network layer after processing by the first network layer; then inputting the basic image features output by the first network layer into the second network layer, and obtaining the basic image features output by the second network layer after processing by the second network layer, and so on, using the output of the previous network layer as the input of the next network layer, and obtaining the output basic image features of the next network layer after processing by the next network layer; wherein, multiple network layers in the basic component are arranged in sequence.

[0084] In some embodiments, the basic component includes multiple network layers, and weighted image features are obtained based on the basic image features and the target attention units, including: obtaining the basic image features output by each network layer in the multiple network layers; performing feature mapping on each basic image feature to obtain multiple target mapping features; obtaining a target attention matrix based on the multiple target mapping features and the target attention units in the target task component; and obtaining weighted image features based on the basic image features and the target attention matrix output by the last network layer of the basic component.

[0085] In some embodiments, obtaining weighted image features based on basic image features and target attention units also includes: obtaining basic image features output by the last network layer in multiple network layers, and obtaining weighted image features based on the basic image features output by the last network layer and the target attention unit.

[0086] In some examples, the image to be recognized is input into the target recognition model, and the basic image features output by the last network layer in the basic component of the target recognition model are obtained through each network layer in the basic component. For example, when the basic component includes four network layers, namely the first to fourth network layers, the image to be recognized can be input into the first network layer to obtain the basic image features output by the first network layer, and the basic image features output by the first network layer can be input into the second network layer to obtain the basic image features output by the second network layer. And so on, the basic image features output by the fourth network layer are obtained. After obtaining the basic image features output by the fourth network layer, the basic image features of the fourth network layer are input into the target attention unit to obtain the weighted image features.

[0087] In some examples, the recognition results of the target objects corresponding to each task type in the image to be identified are determined based on the target task head unit and the weighted image features, including: the target task head unit in the target task component performs recognition calculation on the weighted image features to obtain the recognition results of the target objects in the image to be identified.

[0088] It should be noted that different task types can correspond to the same basic component and different target task components. After the image to be identified is input into the target recognition network, the image to be identified can be processed by the basic component and then input into different target task components. Each target task component will identify the corresponding target object in the image to be identified to obtain the recognition results of each target object.

[0089] The image recognition method provided by the embodiment of the present disclosure inputs the image to be recognized into the target recognition model, and extracts basic image features through the basic component; calculates weighted image features based on the basic image features and the target attention unit; performs recognition operation on the weighted image features through the target task head unit to obtain the recognition result of the target object in the image to be recognized, extracts basic image features through the basic component, calculates the basic image features according to the target attention unit to obtain weighted image features, and processes them through the target task head unit to obtain the recognition result of the target object in the image to be recognized, wherein the basic component obtains shared basic image features, and utilizes the target attention unit and the target task head unit corresponding to the target recognition task to realize the sharing of basic image features output by the basic component by different task types, thereby reducing the consumption of computing resources for image recognition.

[0090] The image recognition method provided by the embodiment of the present disclosure uses sample sets of different recognition tasks to train the recognition model, so that each recognition task complements each other. While learning a recognition task and generating common features, the recognition model can also copy other tasks for learning, reducing the overfitting of the recognition model and improving the accuracy of the trained recognition model in recognizing images. In addition, based on the structural characteristics of the recognition model, when the recognition tasks corresponding to different task types are obtained, the corresponding task components can be obtained based on the task type, and the task components and basic components can be assembled to obtain the target recognition model, that is, by arbitrarily combining models, each task component can realize rapid recognition of different scenes in which the robot is located. Moreover, when performing task recognition on different recognition tasks, different task components can be used for different recognition tasks, and different recognition tasks are decoupled from each other and do not interfere with each other.

[0091] It should be noted that the basic components and task components (such as the attention unit and the task head unit) mentioned in the above embodiments are all pre-trained and stored. The training process of the basic components and task components is described below.

[0092] In some embodiments, before the above-mentioned step 104, the method further includes: obtaining a sample set corresponding to multiple recognition tasks; wherein the sample set includes multiple sample images, and the sample images carry object labels under the corresponding recognition tasks; based on the multiple sample images and the object labels corresponding to each sample image, the initial recognition model is trained to obtain a trained basic component and a trained task component; wherein the initial recognition model includes an initial basic component and initial task components corresponding to different recognition tasks.

[0093] In some examples, a sample set refers to a dataset used to train a model, including multiple sample images that carry object labels corresponding to the recognition task. Sample images refer to images whose parameters can be adjusted for model parameters, and these sample images carry object labels corresponding to the recognition task. An initial recognition model refers to a model that has not been trained on sample sets corresponding to different recognition tasks. Therefore, the initial recognition model cannot perform task recognition on images corresponding to different recognition tasks.

[0094] For example, obtaining a sample set corresponding to a recognition task includes obtaining the sample set from a recognition task sample library, wherein the recognition task sample library stores sample images and corresponding object labels. The object labels may be obtained by manually labeling the collected images.

[0095] Exemplarily, based on multiple sample images and object labels corresponding to each sample image, the initial recognition model is trained to obtain trained basic components and trained task components, including: inputting the sample image into the initial recognition model to obtain the prediction result corresponding to the sample image; based on the prediction result and the object label corresponding to the sample image, the initial recognition model is trained to obtain a trained recognition model; based on the trained recognition model, the trained basic components and trained task components are obtained.

[0096] In some examples, a sample image is input into an initial recognition model to obtain a prediction result corresponding to the sample image, and the initial recognition model is trained based on the prediction result and the object label corresponding to the sample image to obtain a trained recognition model, including: extracting any one of a plurality of sample images, inputting the any one of the sample images into the initial recognition model to obtain a prediction result corresponding to the any one of the sample images, calculating the prediction result and the object label corresponding to the any one of the sample images to obtain a loss value, and adjusting the parameters of the basic components and task components in the initial recognition model based on the loss value to obtain a trained recognition model.

[0097] In some examples, based on the trained recognition model, the trained basic components and the trained task components are obtained, including: based on the preset structure of the recognition model (such as the third preset structure in the above embodiment), the trained recognition model is split to obtain the trained basic components and the trained task components, wherein the preset structure is that the recognition model includes the basic components and the task components.

[0098] It should be noted that the recognition model consists of a basic component and at least one task component. Different recognition tasks require different task components. The number of task components is the same as the number of recognition tasks. Different recognition tasks share the same basic component.

[0099] The image recognition method provided by the embodiments of the present disclosure obtains sample sets corresponding to different recognition tasks, uses multiple sample images and corresponding object labels in the sample sets to train the initial recognition model, and obtains trained basic components and task components, so that image recognition can be performed for different recognition tasks based on the trained basic components and task components, thereby improving the efficiency of subsequent image recognition.

[0100] In some embodiments, the above-mentioned training of the initial recognition model based on multiple sample images and object labels corresponding to each sample image to obtain a trained basic component and a trained task component includes: obtaining a first sample image, wherein the first sample image is any sample image in any sample set; inputting the first sample image into the initial basic component in the initial recognition model to extract the first basic image features; performing a recognition operation on the first basic image features through the initial first task component corresponding to the recognition task of the first sample image to determine a first predicted recognition result of the first sample image; determining a first loss value based on the first predicted recognition result and the first object label, wherein the first object label is the object label carried by the first sample image; adjusting the parameters of the initial basic component and the initial first task component based on the first loss value, and returning to execute the step of obtaining the first sample image until the training stop condition is reached to obtain the trained basic component and the trained first task component.

[0101] In some examples, the first basic image feature refers to a feature obtained by processing the first sample image through a basic component, where the basic component may be a feature extraction layer configured to extract features from the image to obtain the extracted features. The first predicted recognition result refers to a predicted recognition result obtained by predicting the result of the first sample image, where the direction of the prediction is determined based on the recognition task corresponding to the first sample image.

[0102] In some examples, a training stop condition for model training refers to a condition used to determine whether to stop adjusting the model parameters. For example, the training stop condition may include the number of iterations being greater than a threshold, and / or the loss value being less than a preset threshold.

[0103] In some examples, obtaining the first sample image may include: extracting any one sample image from multiple sample images in the sample set as the first sample image, where the first sample image carries a corresponding object label.

[0104] In some examples, inputting the first sample image into the initial basic component in the initial recognition model and extracting the first basic image features can include: inputting the first sample image into the initial recognition model, and obtaining the first basic image features after processing by the initial basic component in the initial recognition model; wherein the initial basic component is used to extract features from the input content, and the dimension of the first basic features can be proportional to the number of network layers in the basic component.

[0105] In some examples, performing a recognition operation on the first basic image feature through the first task component corresponding to the recognition task of the first sample image to determine the first predicted recognition result of the first sample image can include: performing a recognition operation on the first basic image feature through each initial task component in the initial recognition model to obtain a predicted recognition result; based on the initial first task component corresponding to the recognition task of the first sample image, determining the predicted recognition result corresponding to the first task component as the first predicted recognition result.

[0106] In some examples, the first task component corresponding to the recognition task of the first sample image is used to perform a recognition operation on the first basic image feature to determine a first predicted recognition result of the first sample image. It may also include: based on the initial first task component corresponding to the recognition task of the first sample image, determining the initial first task component from the initial recognition model, performing a recognition operation on the first basic image feature through the initial first task component, and obtaining a first predicted recognition result of the first sample image.

[0107] It should be noted that there are many ways to implement the first task component corresponding to the recognition task of the first sample image to perform recognition operations on the first basic image features and determine the first predicted recognition result of the first sample image. The specific method is determined based on actual conditions and is not limited in this disclosure.

[0108] Exemplarily, before performing recognition operations on the first basic image features through the first task component corresponding to the recognition task of the first sample image and determining the first predicted recognition result of the first sample image, the method may also include: performing feature enhancement on the first basic image features through a neck component (such as an initial neck component), and passing the enhanced first basic image features to the initial first task component.

[0109] In some examples, determining a first loss value based on the first predicted recognition result and the first object label includes: calculating the first predicted recognition result and the first object label using a preset loss function to obtain the first loss value. The loss function may be a cross entropy loss function, an L1 norm loss function, a maximum loss function, a mean square error loss function, a logarithmic loss function, etc., which is not limited in this disclosure.

[0110] In some examples, the parameters of the initial basic component and the initial first task component are adjusted based on the first loss value, including: based on the first loss value, the parameters of the initial basic component and the initial first task component in the initial recognition model are adjusted using the gradient descent method.

[0111] The image recognition method provided by the embodiment of the present disclosure uses a sample set to adjust the parameters of the initial recognition model. This can be done by extracting the first sample image and the object label carried by the first sample image in the sample set, adjusting the parameters of the basic component in the initial recognition model and the first task component corresponding to the first sample image, obtaining the adjusted basic component and the first task component, and using the above-mentioned method to adjust the parameters of the initial recognition model so that the adjusted basic component can process different recognition tasks, thereby improving the efficiency of processing the recognition tasks. Moreover, when the same sample image is used as the image corresponding to at least two recognition tasks, the basic component only needs to be processed once, reducing the consumption of computing resources.

[0112] In some embodiments, the task component includes an attention unit and a task head unit; the above steps perform recognition operations on the first basic image features through the initial first task component corresponding to the recognition task of the first sample image to determine the first predicted recognition result of the first sample image, including: determining the initial first task component corresponding to the recognition task of the first sample image; calculating the first weighted image features based on the first basic image features and the initial first attention unit of the initial first task component; and performing recognition calculations on the first weighted image features through the initial first task head unit of the initial first task component to obtain the first predicted recognition result of the first sample image.

[0113] In some embodiments, based on the first loss value, the parameters of the initial basic component and the initial first task component are adjusted, including: based on the first loss value, the parameters of the initial basic component, the initial first attention unit and the initial first task head unit are adjusted.

[0114] In some examples, determining an initial first task component corresponding to a recognition task of a first sample image includes: determining a task type corresponding to the first sample image based on an object label carried by the first sample image, and determining the initial first task component based on the task type corresponding to the first sample image.

[0115] In some examples, a first weighted image feature is calculated based on the first basic image feature and the initial first attention unit in the initial first task component, including: inputting the first basic image feature into the initial first attention unit in the initial first task component to obtain the first weighted image feature, wherein the first attention unit includes an attention matrix corresponding to the first basic image.

[0116] In some examples, the initial first task head unit in the initial first task component performs recognition calculation on the first weighted image feature to obtain a first predicted recognition result for the first sample image, including: inputting the first weighted image feature into the initial first task head unit in the initial first task component to perform recognition calculation to obtain a first predicted recognition result for the first sample image.

[0117] In the image recognition method provided by the embodiment of the present disclosure, when the task component includes an attention unit and a task head unit, the first basic image feature is recognized by the first task component, the first basic image feature is processed by the first attention unit to obtain a first weighted image feature, the first weighted image feature is recognized by the first task head unit to obtain a first predicted recognition result, the first basic image feature is weightedly calculated in the first attention unit, and then the recognition operation is performed using the first weighted image feature obtained by the weighted calculation, that is, the first basic image feature obtained by a basic component can be used to obtain the weighted image feature corresponding to the recognition task using the first attention unit corresponding to the recognition task, thereby realizing the sharing and specificity of the basic image features obtained by the basic component and the reduction of computing resource consumption.

[0118] In some embodiments, the above steps calculate the first weighted image features based on the first basic image features and the initial first attention unit in the initial first task component, including: obtaining the first basic image features output by each network layer of the initial basic component; performing feature mapping on each first basic image feature to obtain a first mapping feature; obtaining a first attention matrix based on the first mapping feature and the initial first attention unit of the initial first task component; and calculating the first weighted image features based on the first basic image features output by the last network layer of the initial basic component and the first attention matrix.

[0119] In some examples, obtaining first basic image features output by each network layer of the initial basic component includes: obtaining each first basic image feature obtained by performing feature extraction processing on the sample image by each network layer in the basic component.

[0120] Exemplarily, the initial basic component includes multiple network layers, which are arranged in sequence. The basic image features output by the previous network layer can be used as input to the next network layer. For example, a sample image can be input to the first network layer among the multiple network layers. After processing by the first network layer, the first basic image features of the first network layer are output. The first basic image features output by the first network layer are then input to the second network layer. After processing by the second network layer, the first basic image features of the second network layer are obtained. This process continues until the first basic image features output by the last network layer in the basic component are obtained.

[0121] In some examples, feature mapping is performed on each first basic image feature to obtain a first mapping feature, including: feature fusing each first basic image feature to obtain a fused feature, and feature mapping the fused feature to obtain a first mapping feature; or, feature mapping is performed on each first basic image feature through a multi-layer perceptron to obtain a first mapping feature.

[0122] In some examples, a first attention matrix is ​​obtained based on the first mapping feature and the initial first attention unit in the initial first task component, including: inputting the first mapping feature into the initial first attention unit of the initial first task component to perform attention calculation to obtain the first attention matrix.

[0123] In some examples, a first weighted image feature is calculated based on a first basic image feature output by the last network layer of the initial basic component and a first attention matrix, including multiplying the first basic feature output by the last network layer of the initial basic component with the first attention matrix to obtain the first weighted image feature.

[0124] It should be noted that after the initial recognition model is trained using the above method, a trained recognition model can be obtained, which includes trained basic components and trained task components corresponding to each task type. The trained recognition model can be split to obtain basic components and multiple task components, and the basic components and multiple task components can be stored. For example, the trained task components can be stored in a component library, that is, the component library includes multiple pre-trained task components. At the same time, the component library can also store the correspondence between each task component and the task type. Therefore, after determining the task type, the target task component corresponding to the task type can be searched in the component library. Different task types correspond to different target task components.

[0125] In some examples, the trained basic components may also be stored, where the number of basic components may be one, and each task type corresponds to one basic component.

[0126] For example, a trained task component may include a trained attention unit and a trained task head unit. That is, the component library stores multiple pre-trained attention units and task head units. For example, one task type may correspond to one attention unit and one task head unit.

[0127] The image recognition method provided by the embodiment of the present disclosure can perform feature mapping on each first basic image feature when calculating the first weighted image feature based on the first basic image feature and the first attention unit of the first task component to obtain the first mapping feature, and use the first attention unit to process the first mapping feature to obtain the first attention matrix, and process the first attention matrix and the first basic image feature outputted by the last layer in the basic component to obtain the first weighted image feature, so that the feature of the subsequent output of the first task head unit is obtained by processing the first basic image feature outputted by the last layer in the basic component with the first attention weight, that is, through the processing of the basic component once and obtaining the corresponding attention weight through the attention unit corresponding to each task component, and performing personalized processing on the output of the basic component to obtain the corresponding output result, thereby reducing the amount of computation consumed in the task recognition processing.

[0128] It should be noted that step 104 above determines the target recognition model when a target task component corresponding to each task type can be found in multiple pre-selected and trained task components. However, if a task type in the recognition task cannot be found in the pre-trained task components, it is necessary to first generate a target task component corresponding to that task type and then determine the target recognition model.

[0129] In some embodiments, if there is no first target task component corresponding to the first task type among multiple pre-trained task components, then based on the first task type, the task component to be trained corresponding to the first task type is determined; wherein, at least one task type includes the first task type; the task component to be trained corresponding to the first task type is trained to obtain the first target task component corresponding to the first task type; and based on the basic component and the first target task component, a target recognition model is determined.

[0130] In some examples, the first task type is at least one task type corresponding to the recognition task, and none of the corresponding task components can be found in the pre-selected trained task components. When the task component corresponding to the first task type cannot be found in the component library, the task component to be trained corresponding to the first task type can be first obtained and added to the target recognition model to obtain an updated target recognition model training; then, the updated target recognition model is trained using the sample set corresponding to the first task type to obtain a trained target recognition model. The trained recognition model includes the trained task component corresponding to the trained first task type, that is, the trained recognition model includes the first target task component corresponding to the first task type. Therefore, the first target task component can be obtained.

[0131] For example, the task component to be trained corresponding to the first task type may also be referred to as the initial task component corresponding to the first task type. The initial task components corresponding to each task type may be pre-stored. When the target task component corresponding to the task type cannot be found, the target task component may be obtained by training the initial task component.

[0132] For example, after obtaining the first target task component, the basic component, the first target task component, and the task components corresponding to other task types can be used to determine the target recognition model. For example, the first target task component and the task components corresponding to other task types can be connected in parallel and then connected in series with the basic component to obtain the target recognition model.

[0133] In some embodiments, the task component to be trained corresponding to the first task type is trained to obtain a first target task component corresponding to the first task type, including: obtaining a first target recognition task corresponding to the first task type, and a target sample set corresponding to the first target recognition task; wherein the target sample set includes multiple target sample images, and the target sample images carry target object labels under the first target recognition task; according to the first target recognition task, the task component to be trained corresponding to the first target recognition task is added to a pre-trained recognition model to obtain an updated recognition model; wherein the pre-trained recognition model includes a pre-trained basic component and pre-trained task components corresponding to different recognition tasks; based on multiple target sample images and target object labels corresponding to the target sample images, the task component to be trained in the updated recognition model is trained to obtain the first target task component.

[0134] In some examples, obtaining a target sample set corresponding to the first target recognition task includes obtaining the target sample set from a recognition task sample library. The recognition task sample library stores sample images and object labels corresponding to the sample images, and the object labels may be obtained by manually labeling the captured images.

[0135] In some examples, based on the first target recognition task, the task component to be trained corresponding to the first target recognition task is added to the pre-trained recognition model to obtain an updated recognition model, including: obtaining the original task component based on the target recognition task, assembling the original task component with the pre-trained recognition model, determining the original task component in the assembled recognition model as the task component to be trained corresponding to the first target recognition task (also referred to as the initial task component), and determining the assembled recognition model as the updated recognition model.

[0136] In some examples, based on multiple target sample images and target object labels corresponding to the target sample images, the task component to be trained in the updated recognition model is trained to obtain a first target task component, including: inputting the target sample image into the updated recognition model to obtain a prediction result corresponding to the target sample image; based on the prediction result and the target object label corresponding to the target sample image, the updated recognition model is trained to obtain a trained recognition model; based on the trained recognition model, the first target task component is obtained; wherein the first target task component is the task component to be trained in the trained recognition model.

[0137] In some examples, the target sample image is input into an updated recognition model to obtain a prediction result corresponding to the target sample image, and the updated recognition model is trained based on the prediction result and the target object label corresponding to the target sample image to obtain a trained recognition model, including: extracting any one of a plurality of target sample images, inputting any one of the target sample images into the updated recognition model to obtain a prediction result corresponding to any one of the target sample images, calculating the prediction result and the target object label corresponding to any one of the target sample images to obtain a loss value, and adjusting the parameters of the initial task component in the updated recognition model based on the loss value to obtain a trained recognition model.

[0138] In some examples, based on the trained recognition model, a trained first target task component is obtained, including: based on a preset structure of the recognition model (such as the third preset mechanism mentioned above), splitting the trained recognition model to obtain a basic component and a first target component, wherein the preset structure is that the recognition model includes a basic component and a task component.

[0139] For example, a pre-trained recognition model may include attention matrices corresponding to three recognition tasks, and the attention matrices corresponding to the three recognition tasks are 3*N matrices, where N represents the number of network layers in the basic component or the number of channels contained in all network layers. If the target task component corresponding to the target recognition task is not found based on the task type, an initial task component corresponding to the target task recognition task is added to the pre-trained recognition model. The initial task component includes an initial attention unit and an initial task head unit, where the initial attention unit includes the initial attention matrix corresponding to the target recognition task, so that the attention matrix in the recognition model becomes a 4*N matrix.

[0140] The image recognition method provided by the embodiments of the present disclosure obtains a target sample set corresponding to a target recognition task, uses multiple sample images and corresponding object labels in the target sample set to train an updated recognition model, and obtains a trained initial task component, so that image recognition can be performed for the target recognition task based on the basic component and the trained initial task component, thereby improving the efficiency of subsequent recognition of the images to be recognized corresponding to the target recognition task.

[0141] In some embodiments, the above steps train the updated recognition model based on multiple target sample images and target object labels corresponding to the target sample images to obtain a first target task component, including: obtaining a second sample image, wherein the second sample image is any target sample image in the target sample set; inputting the second sample image into the basic component in the updated recognition model to extract the second basic image features; performing a recognition operation on the second basic image features through the task component to be trained to determine a second predicted recognition result of the second sample image; calculating a second loss value based on the second predicted recognition result and the second object label, wherein the second object label is the object label carried by the second sample image; adjusting the parameters of the task component to be trained based on the second loss value, and returning to execute the step of obtaining the second sample image until the training stop condition is reached to obtain the trained recognition model; wherein the trained recognition model includes the first target task component.

[0142] In some examples, the second basic image feature refers to a feature obtained by processing the second sample image through a basic component, wherein the basic component can be a feature extraction layer for performing feature extraction on the image to obtain the extracted feature. The second predicted recognition result refers to a predicted recognition result obtained by predicting the result of the second sample image, wherein the direction of the prediction is determined based on the recognition task corresponding to the second sample image. The training stop condition refers to a condition for determining whether the parameter adjustment of the model is stopped. For example, the training stop condition can be that the number of iterations is greater than a threshold number, the loss value is less than a preset threshold, etc.

[0143] In some examples, obtaining the second sample image includes: extracting any one sample image from a plurality of sample images in the target sample set as the second sample image, where the second sample image carries a corresponding object label.

[0144] In some examples, the second sample image is input into the basic component in the updated recognition model to extract the second basic image features, including: inputting the second sample image into the updated recognition model, and obtaining the second basic image features through the basic component in the updated recognition model, wherein the basic component is used to extract features from the input content, and the dimension of the second basic features is proportional to the number of network layers in the basic component.

[0145] In some examples, the second basic image features are recognized through the task component to be trained to determine the second predicted recognition result of the second sample image, including: each task component in the updated recognition model performs a recognition operation on the second basic image features to obtain a predicted recognition result; based on the task component to be trained corresponding to the recognition task of the second sample image, the predicted recognition result corresponding to the task component to be trained is determined as the second predicted recognition result.

[0146] In other examples, the second basic image features are recognized through the task component to be trained to determine the second predicted recognition result of the second sample image, and it also includes: based on the initial task component corresponding to the recognition task of the second sample image, the task component to be trained is determined from the updated recognition model, and the second basic image features are recognized through the task component to be trained to obtain the second predicted recognition result of the second sample image.

[0147] It should be noted that there are many ways to perform recognition operations on the second basic image features through the task component to be trained to determine the second predicted recognition result of the second sample image. The specific methods are determined according to actual conditions and are not limited in this disclosure.

[0148] In some examples, before performing a recognition operation on the second basic image feature through the task component to be trained to determine the second predicted recognition result of the second sample image, the method may also include: performing feature enhancement on the second basic image feature through the neck component to obtain an enhanced second basic image feature.

[0149] For example, the enhanced second basic image features may be transferred to the task component to be trained, and the task component to be trained may perform a recognition operation on the enhanced second basic image features to determine a second predicted recognition result.

[0150] In some examples, a second loss value is calculated based on the second predicted recognition result and the second object label, including: using a preset loss function to calculate the second predicted recognition result and the second object label to obtain a second loss value, wherein the loss function can be a cross entropy loss function, an L1 norm loss function, a maximum loss function, a mean square error loss function, a logarithmic loss function, etc.

[0151] In some examples, adjusting the parameters of the task component to be trained based on the second loss value includes adjusting the parameters of the initial task component based on the second loss value, wherein the adjustment can be performed using a gradient descent method.

[0152] That is to say, the image recognition method provided by the embodiment of the present disclosure, when the target task component corresponding to the first task type (i.e., the first target task component) cannot be found, can assemble the initial task component corresponding to the first task type with the pre-trained basic component to obtain a recognition model to be trained, and train the recognition model to be trained using the sample set corresponding to the first task type to obtain a trained recognition model, which includes the trained initial task component, and the trained initial task component is the first target task component corresponding to the first task type. After obtaining the first target task component, the target recognition model can be determined based on the first target task, and at the same time, the first target task component can also be added to the component library for subsequent use.

[0153] The image recognition method provided by the embodiment of the present disclosure uses the target sample set to adjust the parameters of the updated recognition model. This can be done by extracting the second sample image in the target sample set and the object label carried by the second sample image, adjusting the parameters of the initial task component corresponding to the second sample image in the updated recognition model, obtaining the adjusted initial task component, and using the above-mentioned method to adjust the parameters of the updated recognition model so that the basic component can process the target recognition task, thereby improving the efficiency of processing the target recognition task. Moreover, when the same sample image is used as the image corresponding to at least two recognition tasks, the basic component only needs to be processed once, reducing the consumption of computing resources.

[0154] FIG3A is a schematic diagram of another image recognition method provided by an embodiment of the present disclosure. The image recognition method will be further described below using FIG3A as an example of the application of the image recognition method provided by the present disclosure in a warehouse environment. As shown in FIG3A , the image recognition method includes the following steps:

[0155] Step 302: Obtain a target recognition task, wherein the target recognition task includes a task type and an image to be recognized.

[0156] For example, the task types in the target recognition task may include road marking recognition and pedestrian recognition, and the target recognition task also includes an image to be recognized.

[0157] For example, step 302 may be followed by step 306 .

[0158] Step 304: Obtain sample sets corresponding to different recognition tasks, wherein the sample sets include multiple sample images, and the sample images carry object labels corresponding to the recognition tasks.

[0159] In some examples, sample sets corresponding to different task types, such as road marking recognition, obstacle recognition, and pedestrian recognition, are obtained. Each sample set corresponding to a recognition task includes sample images for that task, with artificially labeled sample objects on the sample images. The sample images in the sample sets corresponding to different recognition tasks overlap.

[0160] Step 306: Using multiple sample images and corresponding object labels, the initial recognition model is trained to obtain trained basic components and task components, wherein the recognition model includes basic components and task components corresponding to different recognition tasks.

[0161] In some examples, an initial recognition model is trained using multiple sample images and corresponding object labels from a sample set corresponding to each recognition task, yielding trained basic components and task components. The recognition model includes the trained basic components and trained task components corresponding to road marking recognition, obstacle recognition, and pedestrian recognition, respectively.

[0162] FIG3B is a schematic diagram of a recognition model training process provided by an embodiment of the present disclosure. As shown in FIG3B , for a first sample image among multiple sample images, the first sample image is input into a basic component, where the basic component includes multiple network layers, for example, the multiple network layers include a first network layer, a second network layer, a third network layer, and a fourth network layer.

[0163] As shown in Figure 3B, the first sample image is input into each network layer in the basic component to obtain the first basic image features output by each network layer. For example, the first sample image is input into the first network layer in the basic component to obtain the first basic image features output by the first network layer; the features output by the first network layer are input into the second network layer to obtain the first basic image features output by the second network layer, and so on, to obtain the first basic image features output by the third and fourth network layers.

[0164] A multi-layer perceptron is used to perform feature mapping on the first basic image features output by each network layer to obtain the first mapping features; the first mapping features are input into the attention unit to obtain the attention matrix processing process, and the attention matrix 1 corresponding to the marking recognition, the attention matrix 2 corresponding to the obstacle recognition and the attention matrix 3 corresponding to the pedestrian recognition are obtained, among which different recognition tasks share the functions that can be realized by the same attention unit.

[0165] Based on the first basic image features and attention matrix output by the fourth network layer of the basic component, a first weighted image feature is calculated, and the first weighted image feature is input into the corresponding task head unit to obtain the prediction result corresponding to the recognition task. For example, based on the first basic image features and attention weight 1 output by the fourth network layer, a weighted image feature for line marking recognition is calculated. The weighted image feature for line marking recognition is passed through task head unit 1 to obtain the line marking recognition prediction result. Based on the first basic image features and attention weight 2 output by the fourth network layer, a weighted image feature for obstacle recognition is calculated. The weighted image feature for obstacle recognition is passed through task head unit 2 to obtain the obstacle recognition prediction result. Based on the first basic image features and attention weight 3 output by the fourth network layer, a weighted image feature for pedestrian recognition is calculated. The weighted image feature for pedestrian recognition is passed through task head unit 3 to obtain the pedestrian recognition prediction result.

[0166] Based on the prediction result and the object label corresponding to the first sample image, a first loss value is calculated, and based on the first loss value, the parameters of the basic component and the first task component in the recognition model are adjusted to obtain the adjusted basic component and the first task component. For example, based on the line marking recognition prediction result and the object label corresponding to the first sample image, the loss value of the line marking recognition is calculated, and based on the loss value of the line marking recognition, the parameters of the basic component and the line marking recognition task component in the recognition model are adjusted to obtain the adjusted basic component and the line marking recognition task component. It should be noted that a similar method can be used to obtain the adjusted basic component, obstacle recognition task component, and pedestrian recognition task component.

[0167] Step 308: Based on the task type, determine whether there is a target task component.

[0168] Among them, the target task component includes the attention unit and the task head unit.

[0169] If the target task component exists, step 310 is executed; if the target task component does not exist, step 320 is executed.

[0170] Step 310: Based on the task type, select a target attention unit and a target task head unit corresponding to the task type from the pre-trained attention units and task head units.

[0171] For example, based on the task types of road marking recognition and pedestrian recognition, target attention units and target task head units corresponding to road marking recognition and pedestrian recognition are selected from pre-trained attention units and task head units.

[0172] Step 312: Assemble the basic components and the target task components to obtain a target recognition model, wherein the basic components and multiple pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic components.

[0173] For example, the target attention units and target task head units corresponding to the basic components, road marking recognition, and pedestrian recognition are assembled to obtain a target recognition model.

[0174] Step 314: Input the image to be recognized into the target recognition model, and extract basic image features through the basic components.

[0175] Step 316: Calculate weighted image features based on the basic image features and the target attention unit.

[0176] Step 318: The target task head unit performs recognition operation on the weighted image features to obtain a recognition result of the target object in the image to be recognized.

[0177] FIG3C is a schematic diagram of an image recognition process provided by an embodiment of the present disclosure. The above steps 314 to 318 are described below with reference to FIG3C.

[0178] As shown in Figure 3C, the image to be recognized is input into the target recognition model. After passing through each network layer in the basic components of the target recognition model, the basic image features output by the last network layer in the basic components are obtained. For example, the image to be recognized is input into the first network layer in the target recognition model to obtain the basic image features output. The basic image features output by the first network layer are then input into the second network layer to obtain the basic image features output. This process continues in this order until the basic image features output by the fourth network layer are obtained.

[0179] Using basic image features and the attention matrices corresponding to line marking recognition and pedestrian recognition, respectively, weighted attention image features corresponding to line marking recognition and pedestrian recognition are obtained. The weighted attention image features corresponding to line marking recognition are input into the task head unit corresponding to line marking recognition to obtain the recognition result of line marking recognition; the weighted attention image features corresponding to pedestrian recognition are input into the task head unit corresponding to pedestrian recognition to obtain the recognition result of pedestrian recognition. For example, the weighted attention image features corresponding to line marking recognition and the basic image features output by the fourth network layer are input into the task head unit 1 corresponding to the line marking recognition task to obtain the recognition result of line marking recognition; the weighted attention image features corresponding to pedestrian recognition and the basic image features output by the fourth network layer are input into the task head unit 3 corresponding to the pedestrian recognition task to obtain the recognition result of pedestrian recognition.

[0180] Step 320: Obtain a target sample set corresponding to the target recognition task, wherein the target sample set includes a plurality of target sample images, and the target sample images carry target object labels under the target recognition task.

[0181] For example, when the task types included in the target recognition task obtained in step 302 are QR code recognition and pedestrian recognition, if the target task component corresponding to QR code recognition is not found in the target recognition model, the target sample set corresponding to QR code recognition is obtained.

[0182] Step 322: According to the target recognition task, an initial task component corresponding to the target recognition task is added to the pre-trained recognition model to obtain an updated recognition model, wherein the recognition model includes a basic component and task components corresponding to different recognition tasks.

[0183] For example, based on the recognition task of QR code recognition, the initial task components corresponding to the QR code recognition task are added to the pre-trained recognition model to obtain an updated recognition model, wherein the task components in the recognition model updated during the training process can only include the initial task components corresponding to the QR code recognition task.

[0184] Step 324: Utilize multiple target sample images and corresponding target object labels to train the updated recognition model to obtain a trained recognition model.

[0185] FIG3D is a schematic diagram of a recognition model update training provided by an embodiment of the present disclosure. The above steps 320 to 324 are described below with reference to FIG3D.

[0186] As shown in Figure 3D, for a sample image among multiple target sample images, the sample image is input into a basic component, which includes multiple network layers. The sample image is input into the first network layer in the basic component to obtain the basic image features output by the first network layer; the basic image features output by the first network layer are input into the second network layer to obtain the basic image features output by the second network layer, and so on, to obtain the basic image features output by the third and fourth network layers; a multi-layer perceptron is used to perform feature mapping on the basic image features output by each network layer in the basic component to obtain mapping features; the mapping features are input into the attention unit to obtain an attention matrix; based on the basic image features output by the fourth network layer of the basic component and the attention matrix, the weighted image features are calculated, and the weighted image features are passed through the task head unit 4 to obtain a prediction result; based on the prediction result and the object label corresponding to the sample image, a loss value is calculated, and based on the loss value, the parameters of the attention unit and the task head unit 4 in the recognition model are adjusted to obtain an adjusted initial task component, wherein the parameters of the multi-layer network layer are frozen.

[0187] Step 326: Input the image to be recognized into the trained recognition model to obtain a recognition result of the target object in the image to be recognized.

[0188] For example, the image to be recognized is input into the trained recognition model to obtain the recognition results of the QR code and the pedestrian in the image to be recognized.

[0189] In an embodiment of the present disclosure, when a target recognition task is obtained, a target task component is obtained according to the task type included in the target recognition task, the target task component is assembled with a basic component shared by different recognition tasks to obtain a target recognition model, and the image to be recognized in the target recognition task is recognized to obtain a recognition result. That is, by using the basic component shared by different recognition tasks, when performing task recognition, the features output by the basic component can be used to perform corresponding processing in the target task component, thereby avoiding repeated consumption of resources used for corresponding processing functions of the basic component when recognizing different recognition tasks, and reducing resource consumption when recognizing recognition tasks.

[0190] Corresponding to the above method embodiment, the present disclosure also provides an embodiment of an image recognition device. FIG4 is a schematic diagram of an image recognition device provided by an embodiment of the present disclosure. As shown in FIG4 , the device includes:

[0191] The acquisition module 402 is configured to: acquire a target recognition task; wherein the target recognition task includes at least one task type and an image to be recognized.

[0192] The determination module 404 is configured to: determine, based on at least one task type, at least one target task component corresponding to each task type from a plurality of pre-trained task components.

[0193] The assembly module 406 is configured to: determine the target recognition model based on assembling basic components and at least one target task component corresponding to each task type; wherein the basic components and multiple pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic components.

[0194] The input module 408 is configured to input the image to be recognized into the target recognition model, and determine the recognition results of the target objects corresponding to each task type in the image to be recognized.

[0195] In some embodiments, the assembly module 406 is further configured to: process multiple target task components in parallel and / or in series to obtain a target task component set; and process the basic component and the target task component set in series to obtain a target recognition model.

[0196] In some embodiments, the target recognition model may further include a neck component. The assembly module 406 is further configured to: connect the base component, the neck component, and the target task component set in series to obtain the target recognition model; wherein the neck component is located between the base component and the target task component set.

[0197] In some embodiments, the assembly module 406 is further configured to: connect in series at least two target task components corresponding to the same task type among the multiple target task components to obtain a target task component subset; and connect in parallel multiple target task component subsets corresponding to different task types to obtain a target task component set.

[0198] In some embodiments, the determination module 404 is further configured to: based on at least one task type, determine the target attention unit and target task head unit corresponding to each task type among multiple pre-trained attention units and multiple task head units; and determine the target task component based on the target attention unit and the target task head unit.

[0199] In some embodiments, the input module 408 is further configured to: input the image to be identified into the basic components in the target recognition model to extract basic image features; obtain weighted image features based on the basic image features and the target attention unit; and determine the recognition results of the target objects corresponding to each task type in the image to be identified based on the target task head unit and the weighted image features.

[0200] In some embodiments, the basic component includes multiple network layers; the input module 408 is further configured to: obtain basic image features output by each of the multiple network layers; perform feature mapping processing on each basic image feature to obtain multiple target mapping features; obtain a target attention matrix based on the multiple target mapping features and the target attention unit; obtain weighted image features based on the basic image features output by the last network layer in the basic component and the target attention matrix.

[0201] In some embodiments, the input module 408 is further configured to: obtain the basic image features output by the last network layer among the multiple network layers; and obtain the weighted image features based on the basic image features output by the last network layer and the target attention unit.

[0202] In some embodiments, the image recognition device also includes a training module, which is configured to: obtain a sample set corresponding to multiple recognition tasks, wherein the sample set includes multiple sample images, and the sample images carry object labels under the corresponding recognition tasks; based on the multiple sample images and the object labels corresponding to each sample image, train the initial recognition model to obtain a trained basic component and a trained task component; wherein the initial recognition model includes an initial basic component and initial task components corresponding to different recognition tasks.

[0203] In some embodiments, the training module includes a first extraction submodule, which is configured to obtain a first sample image, wherein the first sample image is any sample image in any sample set; the training module includes a second extraction submodule, which is configured to input the first sample image into the initial basic component in the initial recognition model to extract the first basic image features; the training module includes a recognition operation submodule, which is configured to perform a recognition operation on the first basic image features through the initial first task component corresponding to the recognition task of the first sample image to determine the first predicted recognition result of the first sample image; the training module includes a calculation submodule, which is configured to determine a first loss value based on the first predicted recognition result and the first object label, wherein the first object label is the object label carried by the first sample image; the training module includes an adjustment submodule, which is configured to adjust the parameters of the initial basic component and the initial first task component based on the first loss value, and return to execute the step of obtaining the first sample image until the training stop condition is reached to obtain the trained basic component and the trained task component.

[0204] In some embodiments, the task component includes an attention unit and a task head unit; the recognition operation submodule is further configured to determine the initial first task component corresponding to the recognition task of the first sample image; the first weighted image feature is calculated based on the first basic image feature and the initial first attention unit in the initial first task component; the first weighted image feature is recognized and calculated by the initial first task head unit in the initial first task component to obtain a first predicted recognition result of the first sample image; the adjustment submodule is further configured to adjust the parameters of the initial basic component, the initial first attention unit and the initial first task head unit based on the first loss value.

[0205] In some embodiments, the recognition operation submodule is further configured to obtain the first basic image features output by each network layer of the basic component; wherein the initial basic component includes multiple network layers; perform feature mapping on each first basic image feature to obtain a first mapping feature; obtain a first attention matrix based on the first mapping feature and the initial first attention unit in the initial first task component; calculate the first weighted image feature based on the first basic image feature output by the last network layer of the basic component and the first attention matrix.

[0206] In some embodiments, the image recognition device also includes a recognition model training module, which is configured to: if there is no first target task component corresponding to the first task type among multiple pre-trained task components, then based on the first task type, determine the task component to be trained corresponding to the first task type; wherein, at least one task type includes the first task type; train the task component to be trained corresponding to the first task type to obtain the first target task component corresponding to the first task type; and determine the target recognition model based on the basic component and the first target task component.

[0207] In some embodiments, the recognition model training module is further configured to: obtain a first target recognition task corresponding to the first task type, and a target sample set corresponding to the first target recognition task; wherein the target sample set includes multiple target sample images, and the target sample images carry the target object label under the target recognition task; according to the first target recognition task, add the task component to be trained corresponding to the first target recognition task to the pre-trained recognition model to obtain an updated recognition model, wherein the pre-trained recognition model includes a pre-trained basic component and pre-trained task components corresponding to different recognition tasks; based on multiple target sample images and target object labels corresponding to the target sample images, train the task component to be trained in the updated recognition model to obtain the first target task component.

[0208] In some embodiments, the recognition model training module is further configured to: obtain a second sample image, wherein the second sample image is any target sample image in the target sample set; input the second sample image into the basic component in the updated recognition model to extract the second basic image features; perform recognition operations on the second basic image features through the initial task component to be trained to determine the second predicted recognition result of the second sample image; calculate a second loss value based on the second predicted recognition result and the second object label; wherein the second object label is the object label carried by the second sample image; based on the second loss value, adjust the parameters of the task component to be trained, and return to execute the step of obtaining the second sample image until the training stop condition is reached to obtain the trained recognition model; wherein the trained recognition model includes the first target task component.

[0209] The image recognition device provided by the embodiment of the present disclosure, when acquiring a target recognition task, obtains a target task component according to the task type included in the target recognition task, assembles the target task component with the basic component shared by different recognition tasks, obtains a target recognition model, recognizes the image to be recognized in the target recognition task, and obtains a recognition result. That is, by using the basic component shared by different recognition tasks, when performing task recognition, the features output by the basic component can be used to perform corresponding processing in the target task component, thereby avoiding repeated consumption of resources used for corresponding processing functions of the basic component when recognizing different recognition tasks, and reducing resource consumption when recognizing recognition tasks.

[0210] The above is a schematic scheme of an image recognition device of this embodiment. It should be noted that the technical solution of the image recognition device and the technical solution of the above-mentioned image recognition method belong to the same concept. For details not described in detail in the technical solution of the image recognition device, please refer to the description of the technical solution of the above-mentioned image recognition method. In addition, the various components in the device embodiment should be understood as functional modules that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0211] Figure 5 is a block diagram of a computing device provided by an embodiment of the present disclosure. As shown in Figure 5, the components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0212] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include any type of network interface, whether wired or wireless, such as one or more network interface controllers (NICs), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0213] In one embodiment of the present disclosure, the aforementioned components of the computing device 500 and other components not shown in FIG5 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG5 is for illustrative purposes only and does not limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed.

[0214] Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 500 may also be a mobile or stationary server.

[0215] The processor 520 is configured to execute computer executable instructions of the image recognition method.

[0216] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned image recognition method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned image recognition method.

[0217] An embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions, which are used in an image recognition method when executed by a processor.

[0218] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-mentioned image recognition method are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-mentioned image recognition method.

[0219] The foregoing description describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0220] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0221] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0222] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0223] The preferred embodiments of the present disclosure disclosed above are intended only to help illustrate the present disclosure. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the present disclosure. The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present disclosure, so that those skilled in the art can better understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. An image recognition method, comprising: Acquire a target recognition task; wherein the target recognition task includes at least one task type and an image to be recognized; Based on at least one of the task types, determining at least one target task component corresponding to each of the task types from a plurality of pre-trained task components; Determine a target recognition model based on a basic component and at least one target task component corresponding to each of the task types; wherein the basic component and the plurality of pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component; The image to be recognized is input into the target recognition model to determine the recognition result of the target object corresponding to each task type in the image to be recognized.

2. The method according to claim 1, wherein: The number of the target task components is multiple; the determining of the target recognition model based on the basic component and the at least one target task component corresponding to each of the task types includes: Processing a plurality of the target task components in parallel and / or in series to obtain a target task component set; The basic component and the target task component set are connected in series to obtain the target recognition model.

3. The method according to claim 2, wherein: The target recognition model may further include a neck component; the target recognition model is determined based on the basic component and at least one of the target task components, including: The basic component, the neck component and the target task component set are connected in series to obtain the target recognition model; wherein the neck component is located between the basic component and the target task component set.

4. The method according to claim 2, wherein: The target task component set is obtained by performing parallel and / or series processing on the plurality of target task components, including: Connecting in series at least two target task components corresponding to the same task type among the plurality of target task components to obtain a target task component subset; A plurality of the target task component subsets corresponding to different task types are connected in parallel to obtain the target task component set.

5. The method according to claim 1, wherein: The determining, based on at least one task type, at least one target task component corresponding to each task type from a plurality of pre-trained task components comprises: Based on at least one of the task types, determining a target attention unit and a target task head unit corresponding to each of the task types from a plurality of pre-trained attention units and a plurality of task head units; wherein the plurality of pre-trained task components include a plurality of pre-trained attention units and a plurality of task head units; Based on the target attention unit and the target task head unit, the target task component is determined.

6. The method according to claim 5, wherein: The step of inputting the image to be recognized into the target recognition model and determining the recognition result of the target object corresponding to each task type in the image to be recognized includes: Inputting the image to be recognized into the basic component in the target recognition model to extract basic image features; Obtaining a weighted image feature according to the basic image feature and the target attention unit; According to the target task head unit and the weighted image features, the recognition result of the target object corresponding to each of the task types in the image to be recognized is determined.

7. The method according to claim 6, wherein: The basic component includes a plurality of network layers; the weighted image features are obtained according to the basic image features and the target attention unit, including: Obtaining basic image features outputted by each of the multiple network layers; Performing feature mapping processing on each of the basic image features to obtain a plurality of target mapping features; Obtaining a target attention matrix according to the plurality of target mapping features and the target attention unit; The weighted image features are obtained according to the basic image features output by the last network layer in the basic component and the target attention matrix.

8. The method according to claim 6, wherein: The basic component includes a plurality of network layers; the weighted image features are obtained according to the basic image features and the target attention unit, including: Obtain basic image features output by a last network layer among the multiple network layers; The weighted image features are obtained according to the basic image features output by the last network layer and the target attention unit.

9. The method according to claim 1, before determining at least one target task component corresponding to each task type from a plurality of pre-trained task components based on at least one of the task types, the method further comprises: Acquire a sample set corresponding to a plurality of recognition tasks; wherein the sample set includes a plurality of sample images, and the sample images carry object labels corresponding to the recognition tasks; Based on the multiple sample images and the object labels corresponding to each sample image, the initial recognition model is trained to obtain a trained basic component and a trained task component; wherein the initial recognition model includes an initial basic component and initial task components corresponding to different recognition tasks.

10. The method according to claim 9, wherein: The initial recognition model is trained based on the multiple sample images and the object labels corresponding to the sample images to obtain the trained basic components and the trained task components, including: Acquire a first sample image; wherein the first sample image is any sample image in any sample set; Inputting the first sample image into the initial basic component in the initial recognition model to extract first basic image features; performing a recognition operation on the first basic image feature through an initial first task component corresponding to the recognition task of the first sample image to determine a first predicted recognition result of the first sample image; Determine a first loss value based on the first predicted recognition result and a first object label; wherein the first object label is an object label carried by the first sample image; Based on the first loss value, the parameters of the initial basic component and the initial first task component are adjusted, and the step of obtaining the first sample image is returned to be executed until the training stop condition is reached, so as to obtain the trained basic component and the trained first task component.

11. The method according to claim 10, wherein: The performing recognition operation on the first basic image feature through the initial first task component corresponding to the recognition task of the first sample image to determine a first predicted recognition result of the first sample image includes: Determining the initial first task component corresponding to the recognition task of the first sample image; Calculating a first weighted image feature according to the first basic image feature and an initial first attention unit in the initial first task component; The initial first task head unit in the initial first task component performs recognition calculation on the first weighted image feature to obtain a first predicted recognition result of the first sample image; The adjusting the parameters of the initial basic component and the initial first task component based on the first loss value includes: Based on the first loss value, parameters of the initial basic component, the initial first attention unit and the initial first task head unit are adjusted.

12. The method according to claim 11, wherein: The step of calculating a first weighted image feature according to the first basic image feature and an initial first attention unit in the initial first task component comprises: Obtaining a first basic image feature output by each network layer of the initial basic component; wherein the initial basic component includes multiple network layers; Performing feature mapping on each of the first basic image features to obtain a first mapping feature; Obtaining a first attention matrix according to the first mapping feature and the initial first attention unit; The first weighted image feature is calculated according to the first basic image feature output by the last network layer in the initial basic component and the first attention matrix.

13. The method according to any one of claims 1 to 12, after acquiring the target recognition task, the method further comprises: If there is no first target task component corresponding to the first task type among the plurality of pre-trained task components, determining a task component to be trained corresponding to the first task type based on the first task type; wherein at least one of the task types includes the first task type; Training the to-be-trained task component corresponding to the first task type to obtain a first target task component corresponding to the first task type; Based on the basic components and the first target task components, the target recognition model is determined.

14. The method according to claim 13, wherein: Training the to-be-trained task component corresponding to the first task type to obtain a first target task component corresponding to the first task type includes: Acquire a first target recognition task corresponding to the first task type and a target sample corresponding to the first target recognition task This set; wherein the target sample set includes a plurality of target sample images, and the target sample images carry the target object label under the first target recognition task; According to the first target recognition task, adding the task component to be trained corresponding to the first target recognition task to the pre-trained recognition model to obtain an updated recognition model; wherein the pre-trained recognition model includes the pre-trained basic component and pre-trained task components corresponding to different recognition tasks; Based on the multiple target sample images and the target object labels corresponding to the target sample images, the task component to be trained in the updated recognition model is trained to obtain the first target task component.

15. The method according to claim 14, wherein: The step of training the task component to be trained in the updated recognition model based on the multiple target sample images and the target object labels corresponding to the target sample images to obtain the first target task component includes: Acquire a second sample image; wherein the second sample image is any target sample image in the target sample set; Inputting the second sample image into the basic component in the updated recognition model, and extracting second basic image features corresponding to the second sample image; Performing a recognition operation on the second basic image feature through the task component to be trained to determine a second predicted recognition result of the second sample image; Calculating a second loss value based on the second predicted recognition result and the second object label; wherein the second object label is an object label carried by the second sample image; Based on the second loss value, the parameters of the task component to be trained are adjusted, and the step of obtaining the second sample image is returned to be executed until the training stop condition is reached to obtain a trained recognition model; wherein the trained recognition model includes the first target task component.

16. An image recognition device, comprising: An acquisition module is configured to acquire a target recognition task; wherein the target recognition task includes at least one task type and an image to be recognized; A determination module is configured to determine, based on at least one of the task types, at least one target task component corresponding to each of the task types from a plurality of pre-trained task components; An assembly module is configured to determine a target recognition model based on a basic component and the at least one target task component corresponding to each of the task types; wherein the basic component and the plurality of pre-trained task components are trained based on task samples of different recognition tasks, and different recognition tasks share the same basic component; The input module is configured to input the image to be recognized into the target recognition model to determine the recognition result of the target object corresponding to each task type in the image to be recognized.

17. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the image recognition method according to any one of claims 1 to 15.

18. A computer-readable storage medium storing computer instructions, wherein the computer instructions, when executed by a processor, implement the image recognition method according to any one of claims 1 to 15.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the image recognition method according to any one of claims 1 to 15.

Images