An image data processing method, a computer device and a readable storage medium

Abstract

Embodiments of the present application provide an image data processing method, a computer device and a readable storage medium, the method relates to the fields of artificial intelligence, intelligent transportation, auxiliary driving and the like, and the method comprises the following steps: obtaining initial image features corresponding to N target images respectively; obtaining S embedding object features associated with target classification label information according to the N initial image features, generating a similarity matrix corresponding to each two embedding object features in the S embedding object features; performing feature enhancement on the S embedding object features based on the similarity matrix corresponding to each two embedding object features, to obtain an enhanced object feature corresponding to each embedding object feature in the S embedding object features respectively; and determining an object recognition result associated with the target classification label information in the N target images according to the enhanced object feature. By using the present application, object recognition can be performed on the N target images, and the accuracy of object recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to an image data processing method, a computer device and a readable storage medium. BACKGROUND

[0002] At present, after a single image (for example, image T) is input into a semantic segmentation model, a convolutional neural network in the semantic segmentation model can perform feature extraction on the image T to obtain a deep feature corresponding to the image T. Further, the semantic segmentation model can generate a superpixel in the image T based on the deep feature corresponding to the image T, determine a superpixel feature corresponding to each superpixel, and then determine an Euclidean distance between the superpixel features, take the Euclidean distance as a feature similarity distance between the superpixel features, and learn a relationship between different superpixels through the feature similarity distance and the superpixel features.

[0003] On the one hand, when performing semantic segmentation on an image, the form of the image is very single, and less information can be used, thereby limiting the accuracy of semantic segmentation. On the other hand, there is a large difference between the classification granularity obtained by the superpixel and the target granularity expected by the semantic segmentation (usually the target granularity is greater than the classification granularity), thereby reducing the accuracy of the semantic segmentation. SUMMARY

[0004] The embodiments of the present application provide an image data processing method, a computer device and a readable storage medium, which can improve the accuracy of object recognition.

[0005] In an aspect, the embodiments of the present application provide an image data processing method, comprising:

[0006] obtaining initial image features corresponding to N target images respectively; the N target images all have the same target classification label information; N is a positive integer greater than 1;

[0007] obtaining S embedded object features associated with the target classification label information according to the N initial image features, and generating a similarity matrix corresponding to each two embedded object features in the S embedded object features; S is a positive integer greater than or equal to N;

[0008] performing feature enhancement on the S embedded object features based on the similarity matrix corresponding to each two embedded object features, to obtain an enhanced object feature corresponding to each embedded object feature in the S embedded object features respectively;

[0009] determining an object recognition result associated with the target classification label information in the N target images according to the enhanced object features.

[0010] In an aspect, the embodiments of the present application provide an image data processing device, comprising:

[0011] obtain initial image features corresponding to N target images respectively; the N target images all have same target classification label information; N is a positive integer greater than 1;

[0012] a matrix generation module, configured to obtain S embedding object features associated with the target classification label information according to the N initial image features, and generate a similarity matrix corresponding to each two embedding object features in the S embedding object features; S is a positive integer greater than or equal to N;

[0013] a feature enhancement module, configured to perform feature enhancement on the S embedding object features based on the similarity matrix corresponding to each two embedding object features, to obtain an enhanced object feature corresponding to each embedding object feature in the S embedding object features respectively;

[0014] an object recognition module, configured to determine an object recognition result associated with the target classification label information in the N target images according to the enhanced object feature.

[0015] The feature acquisition module comprises:

[0016] an image acquisition unit, configured to acquire N target images having same target classification label information, and input the N target images to a target network model; the target network model comprises a first convolutional sub-network configured to perform convolution operation;

[0017] a first convolution unit, configured to perform convolution operation on the N target images through the first convolutional sub-network, to obtain initial image features corresponding to the N target images respectively.

[0018] The target network model further comprises an object detection sub-network configured to perform object detection, a feature embedding sub-network configured to perform feature embedding, and an attention processing sub-network configured to perform attention mechanism processing;

[0019] The matrix generation module comprises:

[0020] an object detection unit, configured to input the N initial image features to the object detection sub-network, and extract S detection object features associated with the target classification label information on the N initial image features through the object detection sub-network; one initial image feature corresponds to one or more detection object features;

[0021] a feature embedding unit, configured to input the S detection object features to the feature embedding sub-network, and perform feature embedding on the S detection object features through the feature embedding sub-network, to obtain S embedding object features having same feature dimension;

[0022] The attention processing unit is used to input S embedded object features into the attention processing sub-network, and to process the S embedded object features through the attention mechanism to generate a similarity matrix corresponding to every two embedded object features.

[0023] Specifically, the object detection unit is used to input N initial image features into the object detection subnetwork. In the object detection subnetwork, object detection is performed on N target images based on the N initial image features, and object detection regions used to characterize the target objects are determined in the N target images. The target objects have target classification label information. The number of object detection regions is S.

[0024] The object detection unit is specifically used to extract the features of the detected objects corresponding to the S object detection regions.

[0025] The attention processing unit is specifically used to input S embedded object features into the attention processing sub-network, and to perform feature operation processing on every two embedded object features in the S embedded object features to obtain the fused object features corresponding to every two embedded object features.

[0026] The attention processing unit is specifically used to determine the feature size of S embedded object features, and to perform feature scaling on the fused object features corresponding to every two embedded object features based on the feature size, so as to obtain the scaled object features corresponding to every two embedded object features.

[0027] The attention processing unit is specifically used to normalize the scaled object features corresponding to each pair of embedded object features, and generate a similarity matrix corresponding to each pair of embedded object features.

[0028] The device also includes:

[0029] The weight generation module is used to take S embedded object features as nodes and generate edge weights between nodes based on the similarity matrix corresponding to each pair of embedded object features; one embedded object feature corresponds to one node; one similarity matrix corresponds to one edge weight.

[0030] The directed graph construction module is used to construct a feature directed graph associated with N target images based on node and edge weights;

[0031] The feature enhancement module is specifically used to enhance the features of S nodes in the directed feature graph based on the edge weights in the directed feature graph, so as to obtain the enhanced object feature corresponding to each of the S embedded object features.

[0032] Among them, the S nodes in the directed graph include node S i; i is a positive integer less than or equal to S; the target network model also includes a feature enhancement subnetwork for feature enhancement;

[0033] The feature enhancement module includes:

[0034] The weight determination unit is used to divide the feature directed graph except for node S. i Nodes other than these are designated as auxiliary nodes;

[0035] The weight determination unit is used to obtain auxiliary nodes for node S from the edge weights through the feature enhancement subnetwork. i The relative edge weights, and node S i Self-edge weights; one auxiliary node corresponds to one relative edge weight;

[0036] The first enhancement unit is used based on node S. i The indicated embedded object features and relative edge weights are used to determine the parameters for node S. i Relative enhancement features for feature enhancement;

[0037] The second enhancement unit is used to acquire node S. i The indicated detection object features are based on node S. i The indicated detection object features and self-edge weights are used to determine the parameters used for node S. i Self-enhancing features that perform feature enhancement;

[0038] The feature aggregation unit is used to aggregate relative enhancement features and self-enhancing features to obtain node S. i The corresponding enhanced object features will be applied to node S. i The corresponding enhanced object features are used as node S i The enhanced object feature corresponding to the indicated embedded object feature.

[0039] The target network model also includes a second convolutional subnetwork for performing convolution operations;

[0040] The object recognition module includes:

[0041] The feature fusion unit is used to acquire the enhanced object features belonging to the same target image, and to perform feature fusion on the enhanced object features belonging to the same target image to obtain the fused image features corresponding to N target images respectively;

[0042] The second convolutional unit is used to input the fused image features into the second convolutional sub-network, and the second convolutional sub-network performs convolution operations on the fused image features to obtain the target image features corresponding to the fused image features;

[0043] The object recognition unit is used to determine the object recognition result associated with the target classification label information in N target images based on the target image features.

[0044] The object recognition unit is specifically used to perform upsampling processing on the target image features to obtain the upsampled image features corresponding to the target image features.

[0045] The object recognition unit is specifically used to perform feature adjustment on the upsampled image features based on the image sizes of N target images to obtain the segmented image features corresponding to the upsampled image features;

[0046] The object recognition unit is specifically used to determine the semantic segmentation results associated with the target classification label information in N target images based on the segmentation image features, and to use the semantic segmentation results as the object recognition results associated with the target classification label information.

[0047] One embodiment of this application provides an image data processing method, including:

[0048] The initial sample features corresponding to N sample images are obtained through the initial network model; all N sample images have the same sample classification label information; N is a positive integer greater than 1;

[0049] In the initial network model, based on N initial sample features, K sample embedding object features associated with sample classification label information are obtained, and a sample similarity matrix is ​​generated for every two sample embedding object features in the K sample embedding object features; K is a positive integer greater than or equal to N;

[0050] Based on the sample similarity matrix corresponding to each pair of sample embedded object features, feature enhancement is performed on K sample embedded object features to obtain the sample enhanced object features corresponding to each sample embedded object feature in the K sample embedded object features;

[0051] Based on the sample augmentation object features and sample classification label information, the parameters of the initial network model are adjusted, and the parameter-adjusted initial network model is used as the target network model; the target network model is used to perform object recognition on the sample objects indicated by the sample classification label information.

[0052] One embodiment of this application provides an image data processing apparatus, including:

[0053] The sample feature acquisition module is used to acquire the initial sample features corresponding to N sample images through the initial network model; all N sample images have the same sample classification label information; N is a positive integer greater than 1;

[0054] The sample matrix generation module is used to obtain K sample embedding object features associated with sample classification label information based on N initial sample features in the initial network model, and generate a sample similarity matrix corresponding to every two sample embedding object features in the K sample embedding object features; K is a positive integer greater than or equal to N;

[0055] The sample feature enhancement module is used to enhance the features of K sample embedding object features based on the sample similarity matrix corresponding to each pair of sample embedding object features, so as to obtain the sample enhanced object feature corresponding to each of the K sample embedding object features;

[0056] The parameter adjustment module is used to adjust the parameters of the initial network model based on the sample augmentation object features and sample classification label information, and use the parameter-adjusted initial network model as the target network model; the target network model is used to perform object recognition on the sample objects indicated by the sample classification label information.

[0057] The initial network model includes a pooling layer for performing pooling operations;

[0058] The parameter adjustment module includes:

[0059] The convolution operation unit is used to obtain the sample augmentation object features belonging to the same sample image, perform feature fusion on the sample augmentation object features belonging to the same sample image, and obtain the sample fused image features corresponding to N sample images respectively;

[0060] The convolution operation unit is used to perform convolution operations on the sample fusion image features to obtain the target sample features corresponding to the sample fusion image features;

[0061] The pooling operation unit is used to input the target sample features into the pooling layer, and the pooling layer performs pooling operations on the target sample features to obtain the pooled feature vector corresponding to the target sample features.

[0062] The normalization processing unit is used to normalize the pooling feature vector to obtain the classification feature vector corresponding to the pooling feature vector; one sample image corresponds to one classification feature vector.

[0063] The parameter adjustment unit is used to adjust the parameters of the initial network model based on the classification feature vector and sample classification label information corresponding to each of the N sample images, and then use the parameter-adjusted initial network model as the target network model.

[0064] The parameter adjustment unit includes:

[0065] The vector determination subunit is used to generate a classification label vector based on the sample classification label information to determine the model loss value of the initial network model;

[0066] The loss determination subunit is used to determine the classification loss value for each sample image based on the classification feature vector and classification label vector corresponding to each of the N sample images.

[0067] The loss determination subunit is used to obtain the model loss value of the initial network model based on the classification loss value corresponding to each sample image.

[0068] The parameter adjustment subunit is used to adjust the parameters of the initial network model based on the model loss value. When the parameter-adjusted initial network model meets the model convergence condition, the parameter-adjusted initial network model is used as the target network model.

[0069] One embodiment of this application provides a computer device, including: a processor and a memory;

[0070] The processor is connected to a memory, which stores a computer program. When the computer program is executed by the processor, it causes the computer device to perform the method provided in the embodiments of this application.

[0071] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided in this application.

[0072] One embodiment of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in this application embodiment.

[0073] In this embodiment, the computer device can acquire initial image features corresponding to N target images, and then acquire S embedded object features associated with target classification label information based on the N initial image features. The N target images all have the same target classification label information, where N can be a positive integer greater than 1, and S can be a positive integer greater than or equal to N. Further, the computer device can generate a similarity matrix corresponding to every two embedded object features among the S embedded object features. Based on the similarity matrix, feature enhancement is performed on the S embedded object features to obtain enhanced object features corresponding to each of the S embedded object features. Further, the computer device can determine the object recognition result associated with the target classification label information in the N target images based on the enhanced object features. Therefore, the computer device in this embodiment can acquire N target images with the same target classification label information at once, acquire S embedded object features associated with the N target images, and perform mutual learning on the S embedded object features to obtain the weight relationship (i.e., similarity matrix) between the S embedded object features. Understandably, the similarity matrix obtained through mutual learning can be used to enhance the features of S embedded object features, resulting in enhanced embedded object features (i.e., enhanced object features). Thus, each of the S enhanced object features incorporates feature information expressed by the other embedded object features; that is, each enhanced object feature incorporates image information from other target images associated with the target classification label information. Based on this, when performing object recognition on each of N target images using the aforementioned enhanced object features, the image information associated with the target classification label information in other target images can be fully utilized, thereby improving the accuracy of object recognition. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0076] Figure 2a This is a schematic diagram of a data interaction scenario provided in an embodiment of this application;

[0077] Figure 2b This is a schematic diagram of a data interaction scenario provided in an embodiment of this application;

[0078] Figure 3 This is a flowchart illustrating an image data processing method provided in an embodiment of this application;

[0079] Figure 4 This is a system framework diagram for object recognition provided in an embodiment of this application;

[0080] Figure 5 This is a flowchart illustrating an image data processing method provided in an embodiment of this application;

[0081] Figure 6 This is a flowchart illustrating an image data processing method provided in an embodiment of this application;

[0082] Figure 7 This is a schematic diagram of a scenario for constructing a directed graph of features, provided in an embodiment of this application.

[0083] Figure 8 This is a flowchart illustrating an image data processing method provided in an embodiment of this application;

[0084] Figure 9 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application;

[0085] Figure 10 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application;

[0086] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0088] It should be understood that Artificial Intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0089] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0090] The embodiments of this invention can be applied to various scenarios, including but not limited to artificial intelligence, intelligent transportation, and assisted driving. Specifically, the solutions provided in this application mainly involve computer vision (CV) and machine learning (ML) technologies in artificial intelligence.

[0091] Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in identifying, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0092] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning. Deep learning, in particular, is a technique that utilizes deep neural network systems for machine learning.

[0093] For details, please see Figure 1 , Figure 1This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, this network architecture may include a service server 2000 and a user terminal cluster. The user terminal cluster may specifically include one or more user terminals; the number of user terminals in the user terminal cluster is not limited here. Figure 1 As shown, the multiple user terminals may specifically include user terminal 3000a, user terminal 3000b, user terminal 3000c, ..., user terminal 3000n; user terminal 3000a, user terminal 3000b, user terminal 3000c, ..., user terminal 3000n can be directly or indirectly connected to the service server 2000 via wired or wireless communication, so that each user terminal can interact with the service server 2000 through the network connection.

[0094] Among them, the business server 2000 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0095] Each user terminal in the user terminal cluster can include: smartphones, tablets, laptops, desktop computers, smart home appliances, wearable devices, in-vehicle terminals, intelligent voice interaction devices, and other intelligent terminals with image data processing capabilities. It should be understood that, for example... Figure 1 Each user terminal in the user terminal cluster shown can be integrated with an application client. When the application client runs on each user terminal, it can interact with the aforementioned... Figure 1 The business servers 2000 shown interact with each other. Specifically, the application clients may include: in-vehicle clients, smart home appliance clients, entertainment clients (e.g., game clients), multimedia clients (e.g., video clients), social networking clients, and information clients (e.g., news clients). The in-vehicle terminal here can be a smart terminal in a smart transportation scenario, and the application clients on this in-vehicle terminal can be the aforementioned in-vehicle clients.

[0096] For ease of understanding, the embodiments of this application may be described in detail below. Figure 1 From the multiple user terminals shown, one user terminal is selected as the target user terminal. For example, in the embodiments of this application, a user terminal can be selected as the target user terminal. Figure 1The user terminal 3000a shown serves as the target user terminal, which may integrate an application client with image data processing capabilities. In this case, the target user terminal can interact with the business server 2000 through this application client.

[0097] It should be understood that the computer device in this application embodiment can obtain N1 target images with the same target classification label information from the target database, generate S enhanced object features associated with the N1 target images, and then determine the object recognition result associated with the target classification label information in the N1 target images based on the S enhanced object features. Here, N1 can be a positive integer greater than 1, and S can be a positive integer greater than or equal to N1. This application embodiment does not limit the specific value of S.

[0098] The object recognition result here can be a semantic segmentation result or an image detection result. In addition, this application can be applied to all problems related to image feature extraction, and is not limited to the above-mentioned semantic segmentation and image detection fields. Therefore, the embodiments of this application do not limit the specific type of object recognition result.

[0099] Similarly, the computer device in this embodiment can obtain N2 sample images with the same sample classification label information from the sample database, generate K sample augmentation object features associated with the N2 sample images through an initial network model, and then perform iterative training (i.e., parameter adjustment) on the initial network model based on the K sample augmentation object features and the sample classification label information, and determine the iteratively trained initial network model as the target network model. Here, N2 can be a positive integer greater than 1, and K can be a positive integer greater than or equal to N2. This embodiment does not limit the specific value of K.

[0100] The target network model is used to perform object recognition on the target image. This object recognition can be semantic segmentation or image detection; the specific type of object recognition is not limited in this embodiment. Semantic segmentation can refer to an algorithm that assigns a pixel category to each pixel in an image, while image detection can refer to an algorithm that extracts and identifies objects in an image using bounding boxes.

[0101] The computer device can be configured with the number of target images N1 and the number of sample images N2. N1 and N2 can be the same positive integer, or they can be different positive integers. This application embodiment does not limit the specific values ​​of N1 and N2. It should be understood that when N1 equals N2, N1 and N2 can be collectively referred to as N. This application embodiment uses N1 equal to N2 as an example for explanation.

[0102] It is understood that the target database may include images with target classification label information, and in this embodiment, images obtained from the target database can be used as target images; the sample database may include images with sample classification label information, and in this embodiment, images obtained from the sample database can be used as sample images. The target classification label information and the sample classification label information can be the same type of classification label information, or they can be different classification label information; the target database and the sample database can be the same database, or they can be different databases.

[0103] The images in the target database and sample database have classification label information. This classification label information can be obtained through manual annotation of the images or through image classification model recognition. It should be understood that the image classification model can learn image information and then identify the image classification. This application does not limit the specific model type of the image classification model.

[0104] It is understood that the image data processing method provided in this application embodiment can be executed by the aforementioned business server 2000 (i.e., the aforementioned computer device can be the business server 2000), or by the aforementioned target user terminal (i.e., the aforementioned computer device can be the target user terminal), or by the business server 2000 and the target user terminal jointly.

[0105] In this image data processing method, when the service server 2000 and the target user terminal are jointly executed, the service server 2000 can iteratively train the initial network model for object recognition based on sample images obtained from the sample database to obtain the target network model. Then, the user corresponding to the target user terminal (e.g., user Y) can send an object recognition request to the service server 2000 through the application client in the target user terminal. This object recognition request can include the target image obtained by user Y from the target database. Further, after receiving the object recognition request, the service server 2000 can obtain the target image from the request, perform object recognition on the target image using the aforementioned target network model, obtain the object recognition result corresponding to the target image, and then return the object recognition result to the target user terminal.

[0106] Optionally, when the image data processing method is executed by the business server 2000, the business server 2000 can directly use the target network model obtained through the above iterative training to perform object recognition on the target image obtained from the target database, and obtain the object recognition result. Optionally, when the image data processing method is executed by the target user terminal, the target user terminal can iteratively train the initial network model used for object recognition based on the sample images obtained from the sample database to obtain the target network model, and then directly use the target network model obtained through iterative training on the target user terminal to perform object recognition on the target image obtained from the target database, and obtain the object recognition result.

[0107] The model training process and the object recognition process mentioned above are both end-to-end (end2end). This end-to-end means that the entire training and prediction processes are complete and coherent, without any interruption in the middle.

[0108] For better understanding, please refer to [link / reference]. Figure 2a and Figure 2b , Figure 2a and Figure 2b This is a schematic diagram illustrating a data interaction scenario provided in an embodiment of this application. For example... Figure 2a and Figure 2b The server 20a shown can be the one described above. Figure 1 The corresponding business server 2000 in the embodiment, such as Figure 2a and Figure 2b The user terminal 20b shown can be the one described above. Figure 1 For ease of understanding, the embodiments of this application refer to any user terminal in the user terminal cluster of the corresponding embodiment. Figure 1 The user terminal 3000a shown is used as an example of the user terminal 20b to illustrate... Figure 2a and Figure 2b The diagram illustrates the specific process of data interaction between server 20a and user terminal 20b. User terminal 20b has an application client installed, which can be used to display target images and object recognition results. The user corresponding to user terminal 20b can be user 20c.

[0109] Among them, it is understandable that, such as Figure 2a The sample database 21a shown may include multiple databases, which may specifically include... Figure 2aThe databases 30a, 30b, ..., 30m are shown. This means that sample database 21a can be used to store sample images corresponding to different classification label information. For example, database 30a can be used to store sample images corresponding to classification label information L1, database 30b can be used to store sample images corresponding to classification label information L2, ..., database 30m can be used to store sample images corresponding to classification label information L3.

[0110] Among them, it is understandable that, such as Figure 2a The target database 21b shown may include multiple databases, which may specifically include... Figure 2a The databases 31a, 31b, ..., 31m are shown. This means that the target database 21b can be used to store target images corresponding to different classification label information. For example, database 31a can be used to store target images corresponding to classification label information L1, database 31b can be used to store target images corresponding to classification label information L2, ..., and database 31m can be used to store target images corresponding to classification label information L3.

[0111] like Figure 2a As shown, server 20a can obtain N sample images with sample classification label information from sample database 21a, where N can be a positive integer. When using classification label information L1 as the sample classification label information, server 20a can obtain N sample images with sample classification label information from database 30a corresponding to classification label information L1, where N can be a positive integer greater than 1. Furthermore, the server can obtain an initial network model for object recognition, and iteratively train the initial network model using the N sample images obtained from sample database 21a, using the iteratively trained initial network model as the target network model.

[0112] like Figure 2a As shown, user 20c can obtain N target images with target classification label information from target database 21b through user terminal 20b. Here, N can be a positive integer. When classification label information L1 is used as the target classification label information, user terminal 20b can obtain N target images with target classification label information from database 31a corresponding to classification label information L1. Specifically, the N target images may include image T1, ..., image T... N .

[0113] Furthermore, when user 20c needs to perform object recognition on N target images, they can send an object recognition request carrying the N target images to server 20a through user terminal 20b. Upon receiving the object recognition request, server 20a can obtain the N target images from the request and perform object recognition on the N target images using the target network model obtained through the iterative training described above, thereby determining the object recognition results associated with the target classification label information in each of the N target images. Here, the object recognition result corresponding to image T1 can be object recognition result G1, ..., image T... N The corresponding object recognition result can be object recognition result G N .

[0114] The specific process by which server 20a performs object recognition on N target images can be found in [reference needed]. Figure 2b ,like Figure 2b As shown, server 20a can obtain initial image features 21c corresponding to N target images through the target network model. One initial image feature in the initial image features 21c corresponds to one target image. Among them, the initial image feature corresponding to image T1 can be initial image feature P1, ..., image T N The corresponding initial image features can be the initial image features P. N .

[0115] Furthermore, server 20a can obtain embedded object features 21d associated with the target classification label information based on the N initial image features 21c. The number of embedded object features in embedded object features 21d is S, where S can be a positive integer greater than or equal to N. It can be understood that one initial image feature corresponds to one or more embedded object features. The embedded object features corresponding to initial image feature P1 can be embedded object features 22a, ..., initial image feature P... N The corresponding embedded object feature can be embedded object feature 22n. Among them, embedded object feature 22a can include F1 embedded object features, ..., embedded object feature 22n can include F... N There are several embedded object features, where F1 can be a positive integer greater than or equal to 1, ..., where F... N It can be a positive integer greater than or equal to 1, where F1, ..., F N The sum can equal S. Where, when F1 equals 1, the F1 embedded object features can include: embedded object feature 23a; ...; in F N When F equals 2, N An embedded object feature may include: embedded object feature 24a and embedded object feature 24b.

[0116] like Figure 2bAs shown, server 20a can generate a similarity matrix corresponding to every two embedded object features among the S embedded object features based on the feature similarity between them. This similarity matrix can include the similarity matrix between each embedded object feature and itself, and can also include the similarity matrix between each embedded object feature and other embedded object features. Specifically, the similarity matrix between embedded object feature 23a and itself can be similarity matrix J1, ...; the similarity matrix between embedded object feature 23a and embedded object feature 24a can be similarity matrix J2; the similarity matrix between embedded object feature 23a and embedded object feature 24b can be similarity matrix J3, ...; and the similarity matrix between embedded object feature 24a and embedded object feature 24b can be similarity matrix J4.

[0117] Furthermore, server 20a can perform feature enhancement on the embedded object features in embedded object feature 21d based on the similarity matrix corresponding to every two embedded object features, obtaining the enhanced object features corresponding to each embedded object feature in embedded object feature 21d. Specifically, server 20a can perform feature enhancement on each embedded object feature based on the similarity matrix associated with each embedded object feature. This similarity matrix associated with each embedded object feature can include the similarity matrix between each embedded object feature and itself, and can also include the similarity matrix between each embedded object feature and other embedded object features.

[0118] It can be understood that the enhanced object feature corresponding to embedded object feature 23a can be enhanced object feature 26a, ..., the enhanced object feature corresponding to embedded object feature 24a can be enhanced object feature 27a, and the enhanced object feature corresponding to embedded object feature 24b can be enhanced object feature 27b. Enhanced object feature 26a can be collectively referred to as enhanced object feature 25a, ..., associated with image T1, and enhanced object features 27a and 27b can be collectively referred to as enhanced object features associated with image T1. N Associated enhanced object features 25n.

[0119] For example, server 20a can perform feature enhancement on embedded object feature 23a based on the similarity matrix associated with embedded object feature 23a to obtain enhanced object feature (i.e., enhanced object feature 26a) corresponding to embedded object feature 23a. The similarity matrix associated with embedded object feature 23a may include the similarity matrix between embedded object feature 23a and itself (e.g., similarity matrix J1), and the similarity matrix associated with embedded object feature 23a may also include the similarity matrix between embedded object feature 23a and other embedded object features (e.g., similarity matrix J2, similarity matrix J3). In other words, server 20a can perform feature enhancement on embedded object feature 23a based on similarity matrix J1, ..., similarity matrix J2 and similarity matrix J3 to obtain enhanced object feature 26a.

[0120] Furthermore, server 20a can determine the object recognition result in the N target images based on the enhanced object features associated with the N target images. Specifically, server 20a can determine the object recognition result (i.e., object recognition result G1) in image T1 based on the enhanced object features 25a associated with image T1 and the target classification label information; ...; server 20a can determine the object recognition result (i.e., object recognition result G1) in image T1 based on the enhanced object features 25a associated with image T1 and the target classification label information; ... N The associated enhanced object features 25n, in image T N The object recognition result (i.e., object recognition result G) is determined by identifying the object recognition result associated with the target classification label information. N ).

[0121] like Figure 2a and Figure 2b As shown, after server 20a performs object recognition on N target images, it can output the object recognition results (i.e., object recognition results G1, ..., object recognition results G) corresponding to the N target images respectively. N The system then returns to user terminal 20b. In this way, user terminal 20b can display the object recognition results corresponding to each of the N target images simultaneously.

[0122] Therefore, this embodiment of the application can acquire N target images with target classification label information at once. Using the target classification label information, it obtains S embedded object features associated with the N target images. These S embedded object features then learn from each other, and the similarity matrix obtained through this mutual learning is used to enhance the features of the S embedded object features. Thus, the enhanced embedded object features (i.e., the enhanced object features) all incorporate feature information expressed by other embedded object features. When using these enhanced object features, which incorporate multiple feature information, to perform object recognition on the N target images, the accuracy of object recognition can be improved.

[0123] Further, please seeFigure 3 , Figure 3 This is a flowchart illustrating an image data processing method provided in an embodiment of this application. The method can be executed by a server, a user terminal, or both. The server can be one of the aforementioned... Figure 2a and Figure 2b The corresponding server 20a in the implementation can be the user terminal described above. Figure 2a and Figure 2b The corresponding user terminal 20b in the implementation. For ease of understanding, this application embodiment uses the method executed by a server as an example for illustration. The image data processing method may include the following steps S101-S104:

[0124] Step S101: Obtain the initial image features corresponding to the N target images respectively;

[0125] Specifically, the server can acquire N target images with the same target classification label information (i.e., all N target images have the same target classification label information) and input these N target images into the target network model. Here, N can be a positive integer greater than 1; the target network model includes a first convolutional sub-network for performing convolution operations. Further, the server can perform convolution operations on the N target images through the first convolutional sub-network to obtain the initial image features corresponding to each of the N target images.

[0126] It should be understood that the first convolutional sub-network can be a CNN (Convolutional Neural Network), which is a basic operational unit in deep learning. This CNN can include convolutional layers, and each convolutional layer can include multiple filters. Convolving an image (e.g., a target image) with a filter generates a feature map corresponding to the image. Convolving this feature map with another filter generates a new feature map, and so on. This application embodiment does not limit the number of filters in the first convolutional sub-network. Here, a filter is a set of convolutional kernels, and the feature map obtained from multiple filters is the aforementioned initial image feature.

[0127] The target image can include multiple channels (for example, when the target image is an RGB (Red, Green, Blue) image, it can include three channels). The filter can perform convolution operations on each of the multiple channels in the target image to obtain channel feature maps corresponding to each channel. Furthermore, after feature fusion of the channel feature maps corresponding to the multiple channels, the feature map corresponding to the target image can be obtained.

[0128] It should be understood that the target network model may further include an object detection sub-network for object detection, a feature embedding sub-network for feature embedding, an attention processing sub-network for attention mechanism processing, a feature enhancement sub-network for feature enhancement, and a second convolutional sub-network for convolutional operations. The target network model is obtained by iteratively training the initial network model. The specific process of the server iteratively training the initial network model to obtain the target network model can be found below. Figure 8 The description of steps S201-S204 in the corresponding embodiment.

[0129] Step S102: Based on N initial image features, obtain S embedded object features associated with the target classification label information, and generate a similarity matrix corresponding to every two embedded object features in the S embedded object features.

[0130] Specifically, the server can input N initial image features into an object detection sub-network, which extracts S detection object features associated with the target classification label information from the N initial image features. Here, one initial image feature corresponds to one or more detection object features. S can be a positive integer greater than or equal to N. Further, the server can input the S detection object features into a feature embedding sub-network, which performs feature embedding on the S detection object features to obtain S embedded object features with the same feature dimension. Further, the server can input the S embedded object features into an attention processing sub-network, which performs attention mechanism processing on the S embedded object features to generate a similarity matrix corresponding to every two embedded object features.

[0131] It is understood that the object detection subnetwork can be an existing weakly supervised object detection model. This existing weakly supervised object detection algorithm can determine one or more detection box regions corresponding to the target object in each of N target images, using target classification label information. Here, each of the one or more detection box regions corresponds to a detected object feature (i.e., one or more detection boxes are mapped to the feature space under CNN). It should be understood that the embodiments of this application do not limit the specific model type of the object detection subnetwork.

[0132] The specific process of object detection performed by the object detection subnetwork can be found below. Figure 5 The description of steps S1021-S1022 in the corresponding embodiment.

[0133] It is understandable that the feature embedding subnetwork can be a CNN (Convolutional Neural Network), through which the convolutional layers of the CNN can map the features of the detected object to the same feature dimension. The feature embedding subnetwork can perform secondary feature embedding on the embedded features (i.e., the features of the detected object), obtaining the corresponding secondary embedded features (i.e., the embedded object features). Therefore, the purpose of secondary embedding is to enable the entire module to form an attention structure (e.g., a Transformer structure, which is a type of algorithm for learning the sequential relationship between sequences), facilitating the subsequent establishment of an attention mechanism between features through the attention processing subnetwork. It should be understood that the embodiments of this application do not limit the specific model type of the feature embedding subnetwork.

[0134] It is understood that the attention processing subnetwork can be a Transformer structure, which can more thoroughly capture the bidirectional relationships in the data. The Transformer structure can include an encoder (Encoder Block) and a decoder (Decoder Block). The encoder can include Multi-Head Attention, Feed Forward, and Add & Norm (Add refers to Residual Connection, Norm refers to Layer Normalization). Multi-Head Attention can include at least one scaled Dot-Product Attention, and the attention processing subnetwork can use a portion of the structure of a scaled dot-product attention as its main framework. It should be understood that the embodiments of this application do not limit the specific model type of the attention processing subnetwork.

[0135] The specific process of attention processing in the attention processing subnetwork can be found below. Figure 5 The description of steps S1023-S1026 in the corresponding embodiment.

[0136] Step S103: Based on the similarity matrix corresponding to each pair of embedded object features, perform feature enhancement on the S embedded object features to obtain the enhanced object features corresponding to each of the S embedded object features.

[0137] Specifically, the server can select from S embedded object features, excluding the embedded object feature S iFeatures other than those of the embedded object are identified as auxiliary embedded object features. Among them, the S embedded object features include embedded object feature S... i Here, i can be a positive integer less than or equal to S. Furthermore, the server can obtain auxiliary embedded object features from the similarity matrix corresponding to every two embedded object features using a feature enhancement subnetwork, targeting the embedded object features S. i The relative similarity matrix and the embedded object features S i The self-similarity matrix is ​​calculated. Each auxiliary embedded object feature corresponds to a relative similarity matrix. Furthermore, the server can further refine the matrix based on the embedded object features S. i And the relative similarity matrix, determine the features S of the embedded object. i The relative enhancement features are used for feature enhancement. Furthermore, the server can obtain the embedded object features S. i The indicated detection object features are based on the embedded object features S. i The indicated detection object features and self-similarity matrix are used to determine the features S of the embedded object. i Self-enhancing features are then used for feature augmentation. Furthermore, the server can perform feature aggregation on the relative enhancement features and the self-enhancing features to obtain the embedded object features S. i The corresponding enhanced object features.

[0138] Updating embedded object features by enhancing object features (i.e., replacing embedded object features with enhanced object features) can be understood as feature enhancement of embedded object features. It can be understood that the enhanced object features used to enhance embedded object features are determined by feature aggregation of the relative enhanced features and the self-enhanced features corresponding to the embedded object features.

[0139] Understandably, the server can perform matrix fusion on the relative similarity matrix to obtain the fused similarity matrix corresponding to the relative similarity matrix, and then apply this to the embedded object features S. i Perform matrix operations on the fusion similarity matrix to obtain the feature S of the embedded object. i Relative enhancement features are those that perform feature enhancement. In other words, the server can enhance the features S of the embedded object. i Matrix operations are performed on the relative similarity matrix to obtain the auxiliary similarity matrix corresponding to the auxiliary embedded object features. Then, matrix fusion is performed on the auxiliary similarity matrix corresponding to each auxiliary embedded object feature to obtain the matrix used for embedding object features S. i Relative enhancement features are used for feature enhancement.

[0140] Understandably, the server can handle the embedded object feature S iMatrix operations are performed on the indicated detection object features and self-similarity matrix to obtain the result used for embedding object features S. i Self-enhancing features that perform feature enhancement.

[0141] Matrix fusion can be an addition operation, which can perform addition on the relative similarity matrix and the auxiliary similarity matrix; matrix operation processing can be a multiplication operation, which can embed object features S. i The multiplication operation is performed with the fusion similarity matrix. This matrix operation can also process the embedded object features S. i Multiplication with the relative similarity matrix is ​​performed, and this matrix operation can also be applied to the embedded object features S. i The indicated detection object features and self-similarity matrix are multiplied; feature aggregation can be an addition operation, which can perform addition operations on relative enhancement features and self-enhancing features.

[0142] It should be understood that the S embedded object features may include the embedded object feature S. i It may also include, in addition to the embedded object feature S i In addition to the (S-1) embedded object features, the server performs feature enhancement on the (S-1) embedded object features to obtain the enhanced object features corresponding to each of the (S-1) embedded object features. For details on this process, please refer to the above description of the embedded object features S. i The description of feature enhancement will not be elaborated here.

[0143] Step S104: Based on the enhanced object features, determine the object recognition results associated with the target classification label information in each of the N target images.

[0144] Specifically, the server can acquire augmented object features belonging to the same target image, perform feature fusion on these augmented object features, and obtain fused image features corresponding to N target images. Further, the server can input the fused image features into a second convolutional sub-network, and perform convolution operations on the fused image features through the second convolutional sub-network to obtain the target image features corresponding to the fused image features. Further, based on the target image features, the server can determine the object recognition results associated with the target classification label information in each of the N target images.

[0145] In this embodiment, when a target image (e.g., target image T) has only one augmented object feature, the server can perform feature fusion on this single augmented object feature to obtain the fused image feature corresponding to target image T. In other words, the server can directly use this single augmented object feature as the fused image feature corresponding to target image T without performing feature fusion on it. It should be understood that the server can perform feature fusion on augmented object features belonging to the same target image in a matrix addition manner or a weighted average manner. This application embodiment does not limit the specific method of feature fusion.

[0146] It is understood that the second convolutional subnetwork can be a CNN (Convolutional Neural Network), which is a basic operational unit in deep learning. This CNN can include convolutional layers, and each convolutional layer can include multiple filters. After fusing image features with multiple filters and performing convolution, target image features can be generated. This application embodiment does not limit the number of filters in the second convolutional subnetwork. The target image features can include deeper information than the fused image features.

[0147] It should be understood that the specific process by which the server determines the object recognition result associated with the target classification label information in N target images based on the target image features can be described as follows: The server can upsample the target image features to obtain upsampled image features corresponding to the target image features. Further, the server can adjust the upsampled image features according to the image sizes of the N target images to obtain segmentation image features corresponding to the upsampled image features. Further, the server can determine the semantic segmentation result associated with the target classification label information in the N target images based on the segmentation image features, and use the semantic segmentation result as the object recognition result associated with the target classification label information.

[0148] The target image features have the ability to locate objects within the target image. To ensure that the feature size of the target image features is as close as possible to the image size of the target image, resulting in more accurate segmentation, the server can upsample the target image features to obtain upsampled image features. At this point, the sizes of the upsampled image features and the target image are not perfectly matched. The server can then obtain a size adjustment function to adjust the feature size. This function is used to adjust the upsampled image features, resulting in segmented image features (i.e., the feature map for semantic segmentation). Here, the feature size of the segmented image features is exactly the same as the image size of the target image.

[0149] Optionally, the server can directly determine the image detection results associated with the target classification label information in N target images based on the target image features, and use the image detection results as the object recognition results associated with the target classification label information.

[0150] For easier understanding, please refer to Figure 4 , Figure 4 This is a system framework diagram for object recognition provided in an embodiment of this application. When this system framework diagram corresponds to the model structure of the target network model, steps S301-S303 in the system framework diagram can correspond to the model application process. For example... Figure 4 As shown, the server can execute step S301, in which multiple images containing objects of the same category are acquired (i.e., N target images with target classification label information). Further, the server can execute step S302, in which feature extraction is performed on the N target images using a convolutional neural network (i.e., the first convolutional sub-network) to obtain the initial image features corresponding to the N target images respectively.

[0151] like Figure 4 As shown, the server can execute step S303, which involves obtaining S detection object features associated with the target classification label information from N initial image features. These S detection object features are then input into a Graph Neural Network (GNN). Based on the S detection object features of the N target images, the GNN generates fused image features corresponding to each of the N target images. Specifically, the server can perform object detection on the N target images using an object detection sub-network to obtain S detection object features, which are then input into the GNN. The GNN outputs target image features corresponding to each of the N target images.

[0152] It can be understood that a graph neural network (Graph Neural Network) can include a feature embedding subnetwork, an attention processing subnetwork, a feature enhancement subnetwork, and a second convolutional subnetwork. S object detection features are sequentially passed through the feature embedding subnetwork, attention processing subnetwork, feature enhancement subnetwork, and second convolutional subnetwork in the Graph Neural Network. The second convolutional subnetwork outputs the target image features corresponding to N target images. Furthermore, the server can determine the object recognition result associated with the target classification label information in each of the N target images based on the target image features.

[0153] Optionally, when N equals 1, the server can obtain the initial image features corresponding to one target image with target classification label information, and then obtain S embedded object features associated with the target classification label information based on the initial image features. Here, S can be a positive integer greater than 1. Further, the server can generate a similarity matrix corresponding to every two embedded object features among the S embedded object features, and perform feature enhancement on the S embedded object features based on the similarity matrix, obtaining enhanced object features corresponding to each of the S embedded object features. Further, the server can perform feature fusion on the S enhanced object features to obtain fused image features corresponding to one target image, and then determine the object recognition result associated with the target classification label information in the aforementioned target image based on the fused image features.

[0154] The specific process of the server acquiring the initial image features corresponding to one target image can be found in the description of step S101 above, and will not be repeated here; the specific process of the server generating S embedded object features of one target image and the similarity matrix corresponding to every two embedded object features in the S embedded object features can be found in the description of step S102 above, and will not be repeated here; the specific process of the server performing feature enhancement on each embedded object feature can be found in the description of step S103 above, and will not be repeated here; the specific process of the server determining the object recognition result associated with the target classification label information in one target image can be found in the description of step S104 above, and will not be repeated here.

[0155] Therefore, the computer device in this embodiment can acquire N target images with the same target classification label information at once, and then acquire S embedded object features associated with the N target images. These S embedded object features are then mutually learned to obtain the weight relationship (i.e., a similarity matrix) between them. It is understood that the similarity matrix obtained based on mutual learning can be used to enhance the S embedded object features, resulting in enhanced embedded object features (i.e., enhanced object features). Thus, each of the S enhanced object features incorporates feature information expressed by other embedded object features; that is, each enhanced object feature incorporates image information associated with the target classification label information from other target images. Based on this, when performing object recognition on each of the N target images using the aforementioned enhanced object features, the image information associated with the target classification label information from other target images can be fully utilized, thereby improving the accuracy of object recognition.

[0156] Further, please see Figure 5 ,Figure 5 This is a flowchart illustrating an image data processing method provided in an embodiment of this application. The image data processing method may include the following steps S1021-S1026, and steps S1021-S1026 are... Figure 3 A specific embodiment of step S102 in the corresponding embodiment.

[0157] Step S1021: Input N initial image features into the object detection sub-network. In the object detection sub-network, perform object detection on N target images based on the N initial image features, and determine the object detection region used to characterize the target object in the N target images.

[0158] The target object has target classification label information, and the number of object detection regions is S.

[0159] Step S1022: Extract the detection object features corresponding to the S object detection regions respectively;

[0160] In this system, one object detection region corresponds to one object detection feature. The server can determine the position of the object detection region in the initial image features by the position of the object detection region in the target image, and then extract the object detection features corresponding to S object detection regions in the initial image features.

[0161] Step S1023: Input the S detection object features into the feature embedding sub-network, and embed the S detection object features through the feature embedding sub-network to obtain S embedded object features with the same feature dimension;

[0162] The specific process of feature embedding subnetwork for embedding features of S detected objects can be found in the above. Figure 3 The description of step S102 in the corresponding embodiments will not be repeated here.

[0163] Specifically, the server can perform attention mechanism processing on every two embedded object features among the S embedded object features through the following steps S1024-S1026, where the S embedded object features may include embedded object features S i and embedded object features S j The server has a specific feature S for the embedded object. i and embedded object features S j The process of attention mechanism processing can be seen in the following formula (1):

[0164]

[0165] in, It can represent the embedded object feature S iThe value of the feature map after the second embedding of features at time t (i.e., the embedded object features) ), It can represent the embedded object feature S j The value of the feature map after the second embedding of features at time t (i.e., the embedded object features) ), It can represent the features of embedded objects. and embedded object features The corresponding similarity matrix, in other words, the embodiments of this application can... As the Q-matrix (i.e., the matrix composed of query vectors) in the Transformer structure, The K matrix (i.e., the matrix composed of key vectors) in the Transformer structure.

[0166] It is understandable that embedded object features and embedded object features The corresponding similarity matrix (i.e.) ) can represent the features of an embedded object The represented object compared to the embedded object features How similar are the objects represented? Optional, embed object features. and embedded object features The corresponding similarity matrix (i.e.) ) can represent the features of an embedded object The represented object compared to the embedded object features How similar are the objects represented? Understandably, this relates to the embedding of object features. and embedded object features When for different embedded object features, the embedded object features and embedded object features The corresponding similarity matrix is ​​a relative similarity matrix. In this case, the embedded object features S... i and embedded object features S j It can be any two embedded object features from S embedded object features. Optionally, in the embedded object features and embedded object features When the features are the same embedded object features, the embedded object features and embedded object features The corresponding similarity matrix is ​​the self-similarity matrix.

[0167] Step S1024: Input the S embedded object features into the attention processing sub-network, and perform feature operation processing on every two embedded object features in the S embedded object features through the attention processing sub-network to obtain the fused object features corresponding to every two embedded object features.

[0168] The feature operation processing can be a multiplication operation, which can perform a multiplication operation on every two embedded object features. For example, this feature operation processing can multiply the embedded object features... and embedded object features Perform multiplication. Specifically, the server analyzes the embedded object's features. and embedded object features Before performing multiplication, it is also necessary to analyze the embedded object features. Perform a transpose operation.

[0169] Step S1025: Determine the feature size of S embedded object features. Based on the feature size, perform feature scaling on the fused object features corresponding to every two embedded object features to obtain the scaled object features corresponding to every two embedded object features.

[0170] Understandably, the server can determine the feature sizes of S embedded object features. Based on these feature sizes, it determines the square root feature size used for feature scaling of the fused object features corresponding to every two embedded object features, and then performs feature scaling on the fused object features based on the square root feature size. The S embedded object features have the same feature size; for example, the feature size can be equal to 4. For instance, when the feature dimension of an embedded object feature is M1*M2 (e.g., 3*4), the feature size of that embedded object feature can be equal to 4. Both M1 and M2 can be positive integers.

[0171] Step S1026: Normalize the scaled object features corresponding to each pair of embedded object features to generate a similarity matrix corresponding to each pair of embedded object features.

[0172] The server can use a normalization exponent function to normalize the features of the scaled object. Here, the normalization exponent function can be the softmax function.

[0173] Among them, the server has embedded object features and embedded object features The specific process of attention mechanism processing can be found in the following formula (2):

[0174]

[0175] in, It can represent S embedded object features (e.g., embedded object features) The feature size (i.e., the size of the feature) Represents the features of the embedded object Characteristics of the transposed object obtained after transposition. It can represent the features of the embedded object. and transpose object characteristics The features of the fused object obtained by performing feature operations. This can represent the square root feature size used for feature scaling of the fused object features (i.e., the square root operation of the fused object features). Therefore, It can represent the features of embedded objects. and embedded object features Corresponding scaling object characteristics, (Right now This can represent the embedded object features obtained by normalizing the scaled object features. and embedded object features The corresponding similarity matrix.

[0176] Therefore, this application embodiment can use a global Transformer structure targeting different locations and semantic representations to calculate the feature relationships (i.e., similarity matrices) between embedded object features, making the relationships between embedded object features more robust. Furthermore, this application embodiment can express detection boxes of the same granularity (i.e., determining object detection regions for representing target objects in N target images based on target classification label information) as embedded object features, thereby achieving mutual learning between embedded object features and ensuring consistency in segmentation granularity.

[0177] Further, please see Figure 6 , Figure 6 This is a schematic flowchart of an image data processing method provided in an embodiment of this application. The image data processing method may include the following steps S1031-S1037, and steps S1033-S1037 are... Figure 3 A specific embodiment of step S103 in the corresponding embodiment.

[0178] Step S1031: Using the S embedded object features as nodes, generate the edge weights between nodes based on the similarity matrix corresponding to each pair of embedded object features.

[0179] Understandably, the server can directly use the similarity matrix corresponding to each pair of embedded object features as the edge weight between nodes. Here, one embedded object feature corresponds to one node, and one similarity matrix corresponds to one edge weight.

[0180] Optionally, the server can determine the contribution information indicated by the similarity matrix corresponding to each pair of embedded object features. Based on the contribution information, it filters the similarity matrices that meet the contribution criteria from the similarity matrices corresponding to each pair of embedded object features, and determines the similarity matrices that meet the contribution criteria as auxiliary similarity matrices. Further, the server can use the auxiliary similarity matrix as the edge weights between nodes. If the weight between two nodes is too low, it can be understood that the contribution of the edge weight is too small, which is equivalent to no edge. If the numbers in a matrix are all small values ​​(e.g., 0.01, 0.02), it can indicate that the contribution information corresponding to that matrix is ​​small.

[0181] It should be understood that in the embodiments of this application, the feature map (i.e. the embedded object feature) of each detection box region obtained by the object detection sub-network after secondary embedding can be used as the node feature, and the node feature can be used as a node in the graph network. The node can represent the feature of the object in the detection box region. Therefore, in the embodiments of this application, in the following step S1032, a graph network can be established with the object as the node (e.g., the detection box region corresponding to the airplane) and the similarity between objects (i.e., the edge weight) as the edge.

[0182] Step S1032: Based on node and edge weights, construct a directed feature graph associated with N target images;

[0183] Graphs (e.g., directed graphs with features) are the most direct tool for describing community relationship chains. They consist of nodes and edges (i.e., edge weights). Nodes represent relationship chain objects, and edges represent the degree of connection between two objects. An edge in a directed graph with features can represent the degree of connection between one node and another node. An edge in a directed graph with features represents a directed edge from one node to another.

[0184] It should be understood that learning a mutual attention mechanism between the secondary embedding features (i.e., the features of the embedded objects) of different nodes can yield the weight values ​​of the edges between different nodes (here, the similarity matrix between the two nodes constituting the edge can be used to represent the edge weight value). The attention mechanism here can use the Transformer module mentioned above. The edge features learned under the attention mechanism can be understood as the similarity between two features.

[0185] For example, if two airplanes have very similar regional features (i.e., embedded object features), then the weight of the edge between the airplane's regional nodes will be large. Conversely, if one region is an airplane and the other is the sky, then the similarity between the airplane and the sky is small, and the weight of the edge between these two nodes will be relatively small. Therefore, similar regional features can mutually reinforce each other, resulting in better object recognition results and thus improving the accuracy of object recognition.

[0186] It should be understood that the specific process by which the server performs feature enhancement on S nodes in the directed feature graph based on the edge weights of the graph, obtaining the enhanced object feature corresponding to each of the S embedded object features, can be described as steps S1033-S1037 below. Here, the S nodes in the directed feature graph include node S... i Here, i can be a positive integer less than or equal to S, and S is the node. i The indicated embedded object feature is the aforementioned embedded object feature S. i .

[0187] Step S1033: Remove node S from the directed graph of features. i Nodes other than these are designated as auxiliary nodes;

[0188] Step S1034: Obtain auxiliary nodes for node S in the edge weights through the feature enhancement subnetwork. i The relative edge weights, and node S i Self-edge weights;

[0189] In this context, an auxiliary node corresponds to a relative edge weight, and a node S i Each corresponds to a self-edge weight. The relative edge weight corresponding to the auxiliary node is the embedding object feature indicated by the auxiliary node and node S. i The similarity matrix (i.e., relative similarity matrix) between the indicated embedded object features, node S i The weight of the self-edge is the node S. i The indicated embedded object features and node S i The similarity matrix (i.e., the self-similarity matrix) between the features of the indicated embedded objects.

[0190] Step S1035, based on node S i The indicated embedded object features and relative edge weights are used to determine the parameters for node S. i Relative enhancement features for feature enhancement;

[0191] Among them, the server determines the node S. i For the specific process of performing relative enhancement of features, please refer to the above description of determining the features S used for embedding objects. i The description of the relatively enhanced features for feature enhancement will not be elaborated here.

[0192] Step S1036, obtain node S i The indicated detection object features are based on node S. i The indicated detection object features and self-edge weights are used to determine the parameters used for node S. iSelf-enhancing features that perform feature enhancement;

[0193] Among them, the server determines the node S. i For the specific process of performing self-enhancing features for feature enhancement, please refer to the above description of determining the features S used for embedding objects. i The description of self-enhancing features for feature augmentation will not be elaborated here.

[0194] Step S1037: Perform feature aggregation on the relative enhancement features and self-enhancing features to obtain node S. i The corresponding enhanced object features will be applied to node S. i The corresponding enhanced object features are used as node S i The enhanced object feature corresponding to the indicated embedded object feature.

[0195] Among them, the server supports node S i The specific process of feature enhancement can be found in the following formula (3):

[0196]

[0197] in, Node S can be represented i The indicated embedded object features, Node S can be represented i The corresponding relative edge weights, Node S can be represented i The indicated detection object features (i.e., node S) i (The value of the feature map after embedding at time t), Node S can be represented i The corresponding self-edge weights. Therefore, Node S can be represented i The relative enhancement feature at time t Node S can be represented i The self-enhancing feature corresponding to time t (Right now ) can represent node S i The enhanced object features at time (t+1). Specifically, the node features at time (t+1) are updated compared to time t.

[0198] It should be understood that the S nodes in a directed graph with features can also include node Si. j Here, j can be a positive integer less than or equal to S, and j is not equal to i. Here, node S... j The indicated embedded object feature is embedded object feature S. j Among them, node S j This can be any S nodes in a directed graph except for node S.i For any node other than S, the server can access that node. j For the specific process of feature enhancement, please refer to the above description of node S. i The description of feature enhancement will not be elaborated here.

[0199] For easier understanding, please refer to Figure 7 , Figure 7 This is a schematic diagram illustrating a scenario for constructing a directed graph of features, as provided in an embodiment of this application. For example... Figure 7 The images 80a, 80b, ..., 80c shown can be target images with the same target classification label information obtained by the server. For example, the target classification label information here can be an airplane.

[0200] like Figure 7 As shown, the server can input images 80a, 80b, ..., 80c into a convolutional neural network (i.e., the first convolutional sub-network). The convolutional neural network extracts features from images 80a, 80b, ..., 80c, and segments images 80a, 80b, ..., 80c to obtain detection boxes associated with the target classification label information (i.e., airplane). For example, the detection box segmented in image 80a can be detection box 81a, the detection box segmented in image 80b can be detection box 81b, and the detection boxes segmented in image 80d (not shown in the figure) can be detection boxes 81c and 81d.

[0201] like Figure 7 As shown, the server can perform secondary embedding on the node features corresponding to the above detection boxes, using the secondary embedded node features as nodes, determining the weights between the secondary embedded node features through a Transformer structure, and using these weights as edges between nodes to generate... Figure 7 The feature is shown as a directed graph. For example, detection box 81a can correspond to node H′i, detection box 81b can correspond to node H′j1, detection box 81c can correspond to node H′j2, and detection box 81d can correspond to node H′j3.

[0202] like Figure 7 As shown, the server can integrate information from nodes and edge weights in a directed feature graph. The purpose of this integration is to establish similarity relationships between nodes, that is, to integrate edge weight information into node features. Node features are influenced by the edge connection weights of surrounding nodes, resulting in updated node information to establish information connections between node features. Furthermore, after the updated node information passes through a convolutional layer (i.e., the second convolutional subnetwork), it can have the ability to locate objects, and thus object recognition can be achieved based on the features output by the convolutional layer.

[0203] Therefore, this application embodiment can acquire N target images with the same target classification label information. Using the positional information of the target object indicated by the target classification label information at different locations in the N target images as nodes, and the relationships between target objects as edge weights, a directed feature graph is constructed. Then, through the directed feature graph, semantic connections between nodes are mined. By performing mutual feature enhancement on the nodes in the directed feature graph, the richness of information can be increased. Furthermore, since embedded object features can represent the manifestation of the target object indicated by the target classification label information in the N target images, the directed feature graph can include various forms displayed by the target object.

[0204] Further, please see Figure 8 , Figure 8 This is a flowchart illustrating an image data processing method provided in an embodiment of this application. The method can be executed by a server, a user terminal, or both. The server can be one of the aforementioned... Figure 2a and Figure 2b The corresponding server 20a in the implementation can be the user terminal described above. Figure 2a and Figure 2b The corresponding user terminal 20b in the implementation. For ease of understanding, this application embodiment uses the method executed by a server as an example for illustration. The image data processing method may include the following steps S201-S204:

[0205] Step S201: Obtain the initial sample features corresponding to N sample images through the initial network model;

[0206] It should be understood that the initial network model may include an initial first convolutional subnetwork for performing convolution operations, an initial object detection subnetwork for performing object detection, an initial feature embedding subnetwork for performing feature embedding, an initial attention processing subnetwork for performing attention mechanism processing, an initial feature enhancement subnetwork for performing feature enhancement, and an initial second convolutional subnetwork for performing convolution operations. Optionally, the initial network model may also include pooling layers for performing pooling operations.

[0207] Specifically, the server can acquire N sample images with the same sample classification label information (i.e., all N sample images have the same sample classification label information) and input these N sample images into the initial network model. Here, N can be a positive integer greater than 1. Further, the server can perform convolution operations on the N sample images through the initial first convolutional subnetwork in the initial network model to obtain the initial sample features corresponding to the N sample images.

[0208] The specific process of the server performing convolution operations on N sample images through the initial first convolutional subnetwork can be found in the description of performing convolution operations on N target images through the first convolutional subnetwork above, and will not be repeated here.

[0209] Step S202: In the initial network model, based on N initial sample features, obtain K sample embedding object features associated with sample classification label information, and generate a sample similarity matrix corresponding to every two sample embedding object features in the K sample embedding object features.

[0210] Specifically, the server can input N initial sample features into an initial object detection sub-network, which extracts K sample detection object features associated with the sample classification label information from the N initial sample features. Here, one initial sample feature corresponds to one or more sample detection object features, and K can be a positive integer greater than or equal to N. Further, the server can input the K sample detection object features into an initial feature embedding sub-network, which performs feature embedding on the K sample detection object features to obtain K sample embedded object features with the same feature dimension. Further, the server can input the K sample embedded object features into an initial attention processing sub-network, which performs attention mechanism processing on the K sample embedded object features to generate a sample similarity matrix corresponding to every two sample embedded object features.

[0211] The specific process by which the server extracts K sample object features from N initial sample features can be found in the description of extracting S detection object features from N initial image features, and will not be repeated here. Similarly, the specific process by which the server embeds features from the K sample object features can be found in the description of embedding features from the S detection object features, and will not be repeated here. Finally, the specific process by which the server applies an attention mechanism to the K embedded object features can be found in the description of applying an attention mechanism to the S embedded object features, and will not be repeated here.

[0212] Step S203: Based on the sample similarity matrix corresponding to each pair of sample embedded object features, perform feature enhancement on K sample embedded object features to obtain the sample enhanced object features corresponding to each sample embedded object feature in the K sample embedded object features.

[0213] It is understandable that the server can perform feature enhancement on the K sample embedded object features through the initial feature enhancement subnetwork. The specific process of the server performing feature enhancement on the K sample embedded object features can be found in the description of feature enhancement on the S embedded object features above, and will not be repeated here.

[0214] Step S204: Based on the sample augmentation object features and sample classification label information, adjust the parameters of the initial network model and use the adjusted initial network model as the target network model.

[0215] Specifically, the server can acquire the augmented object features belonging to the same sample image, perform feature fusion on these features, and obtain sample fused image features corresponding to N sample images. Further, the server can perform convolution operations on the sample fused image features to obtain the target sample features corresponding to the sample fused image features. Further, the server can input the target sample features into a pooling layer, and perform pooling operations on the target sample features to obtain the pooled feature vector corresponding to the target sample features. Further, the server can normalize the pooled feature vector to obtain the classification feature vector corresponding to the pooled feature vector. Here, one sample image corresponds to one classification feature vector. Further, the server can adjust the parameters of the initial network model based on the classification feature vector and sample classification label information corresponding to each of the N sample images, and use the parameter-adjusted initial network model as the target network model.

[0216] Understandably, the server can input the sample fused image features into the initial second convolutional subnetwork, and then perform a convolution operation on the sample fused image features through the initial second convolutional subnetwork to obtain the target sample features corresponding to the sample fused image features. The specific process of the server performing the convolution operation on the sample fused image features through the initial second convolutional subnetwork can be found in the description of performing the convolution operation on the fused image features through the second convolutional subnetwork described above, and will not be repeated here.

[0217] Understandably, the server can perform pooling operations (e.g., average pooling) on ​​the target sample features using pooling layers to reduce the dimensionality of the target sample features, resulting in the dimensionality-reduced target sample features (i.e., the pooled feature vector). This pooled feature vector is one-dimensional, and this one-dimensional vector can be used to represent the probability distribution of a sample image belonging to each sample classification label. Based on this probability distribution, it can be determined which predicted classification label the sample image belongs to, and then, based on this predicted classification label and the sample classification label, the object recognition capability of the initial network model can be determined.

[0218] One sample image can correspond to one pooling feature vector. For example, the pooling feature vector corresponding to sample image T can be pooling feature vector C. When the number of sample classification label information is 2 (e.g., sample classification label information L1 and sample classification label information L2), the pooling feature vector C can include 2 elements. One element is used to represent the probability O1 of sample image T belonging to sample classification label information L1 (e.g., 0.25), and the other element is used to represent the probability O2 of sample image T belonging to sample classification label information L2 (e.g., 0.50).

[0219] The target network model can be used to identify the sample objects indicated by the sample classification label information. In other words, the target network model can perform object recognition on the target image based on the sample classification label information, obtaining the object recognition result corresponding to the target image. The classification label information of the target image can be the target classification label information.

[0220] Understandably, the server can train an initial network model using sample images with sample classification label information to obtain a target network model. This target network model has the function of object recognition based on sample classification label information. Thus, the target network model trained using sample classification label information can be used to perform object recognition on target images with target classification label information. Here, the target classification label information can be the same as the sample classification label information.

[0221] It should be understood that the specific process by which the server adjusts the parameters of the initial network model based on the classification feature vector and classification label information corresponding to each of the N sample images can be described as follows: The server can generate a classification label vector to determine the model loss value of the initial network model based on the sample classification label information. Further, the server can determine the classification loss value corresponding to each sample image based on the classification feature vector and classification label vector corresponding to each of the N sample images. Further, the server can obtain the model loss value of the initial network model based on the classification loss value corresponding to each sample image. Further, the server can adjust the parameters of the initial network model based on the model loss value. When the parameter-adjusted initial network model meets the model convergence condition, the parameter-adjusted initial network model is used as the target network model.

[0222] The classification label vector is determined by the sample classification label information, and its dimension is determined by the number of sample classification labels. When the number of sample classification labels is L, the classification label vector has L dimensions. For example, when the sample classification label information of a sample image is L1, the classification label vector can be (1,0,…,0); when the sample classification label information of a sample image is L2, the classification label vector can be (0,1,…,0).

[0223] Understandably, the server can input the classification feature vector and classification label vector corresponding to each sample image into the model loss function, and determine the model loss value of the initial network model through the model loss function. Here, the model loss value is obtained by performing loss fusion (e.g., weighted averaging) on ​​the classification loss values ​​corresponding to each sample image.

[0224] For better understanding, please refer to [link / reference]. Figure 7 ,like Figure 7 The images 80a, 80b, ..., 80c shown can be sample images with the same sample classification label information obtained by the server. For example, the sample classification label information here can be an airplane.

[0225] like Figure 7 As shown, the server can input images 80a, 80b, ..., 80c into a convolutional neural network (i.e., the initial first convolutional sub-network). The convolutional neural network extracts features from images 80a, 80b, ..., 80c, and segments images 80a, 80b, ..., 80c to obtain detection boxes associated with the sample classification label information (i.e., airplane). For example, the detection box segmented in image 80a can be detection box 81a, the detection box segmented in image 80b can be detection box 81b, and the detection boxes segmented in image 80d (not shown in the figure) can be detection boxes 81c and 81d.

[0226] like Figure 7 As shown, the server can perform secondary embedding on the node features corresponding to the above detection boxes, using the secondary embedded node features as nodes, determining the weights between the secondary embedded node features through a Transformer structure, and using these weights as edges between nodes to generate... Figure 7 The feature is shown as a directed graph. For example, detection box 81a can correspond to node H′i, detection box 81b can correspond to node H′j1, detection box 81c can correspond to node H′j2, and detection box 81d can correspond to node H′j3.

[0227] like Figure 7As shown, the server can integrate information from the nodes and edge weights in the directed feature graph, outputting updated node information to establish information connections between node features. Furthermore, after the updated node information passes through a convolutional layer (i.e., the initial second convolutional sub-network), it can be processed by an average pooling layer to output a category feature vector. This category feature vector is then compared with the image list label vector (i.e., the classification label vector) using a cross-entropy loss function for training.

[0228] It should be understood that the loss function in the model training process can be used to represent the degree of difference between the predicted value and the actual value. The smaller the loss value corresponding to the loss function (e.g., classification loss value, model loss value), the better the model. Therefore, the goal of training a machine learning model is to find the point where the model loss function reaches its minimum value. The model loss function can be a cross-entropy loss function or a logistic loss function. Here, we take the cross-entropy loss function as an example for illustration. This application does not limit the specific type of model loss function.

[0229] Where the sample classification label information of the sample image can be divided into two categories (for example, the sample classification label information of the sample image can be car or airplane, that is, the predicted value can be divided into car (i.e., positive) or airplane (i.e., negative)), this application can use binary cross-entropy as the model loss function of the initial network model, that is, the model loss function can be referred to in the following formula (4):

[0230] C=-(1 / n)∑[ylna+(1-y)ln(1-a)] (4)

[0231] Here, n can be the number of sample images, y can be the actual value of the sample classification label information (when the actual value is a car, y can be 1 (i.e., the classification label vector is equal to (1,0)); when the actual value is an airplane, y can be 0 (i.e., the classification label vector is equal to (0,1)), and a can be the probability of the predicted value being a car (i.e., positive).

[0232] Optionally, when the sample classification label information of the sample image can be divided into multiple classes (for example, the sample classification label information of the sample image can be car, airplane, sky, apple, etc.), this application can use multi-class cross-entropy as the model loss function of the initial network model, that is, the model loss function can be found in the following formula (5):

[0233]

[0234] Where n can be the number of sample images, m can represent the number of sample classification label information, and y cThis can represent an indicator variable (the indicator variable can be 1 when the actual value is the same as the sample classification label information c; the indicator variable can be 0 when the actual value is different from the sample classification label information c), where a c This can represent the probability that the predicted value is the sample classification label information c. The classification label vector is determined by the set of indicator variables mentioned above. For example, the classification loss value for sample image T1 can be: loss1 = -(0×ln0.3 + 0×ln0.3 + 1×ln0.4) = 0.91, the classification loss value for sample image T2 can be: loss2 = -(0×ln0.3 + 0×ln0.3 + 1×ln0.4) = 0.91, and the classification loss value for sample image T3 can be: loss3 = -(0×ln0.7 + 0×ln0.2 + 1×ln0.1) = 2.30. Therefore, the model loss value for all sample images can be: C = (1 / 3)×(0.91 + 0.91 + 2.30) = 1.37. The classification label vector can be represented as (0,0,1).

[0235] It should be understood that the specific process of the server adjusting the parameters of the initial network model based on the model loss value (i.e., iteratively training the initial network model) can be described as follows: When the model loss value of the initial network model does not meet the model convergence condition, the server can adjust the model parameters of the initial network model based on the model loss value that does not meet the model convergence condition. Furthermore, the server can determine the initial network model after adjusting the model parameters as a transitional network model, and iteratively train the transitional network model until the model loss value of the iteratively trained transitional network model meets the model convergence condition. The transitional network model that meets the model convergence condition is then used as the target network model.

[0236] For better understanding, please refer to [link / reference]. Figure 4 When the system framework diagram corresponds to the model structure of the initial network model, steps S301-S305 in the system framework diagram can correspond to the model training process. For example... Figure 4 As shown, the server can execute step S301, in which multiple images containing objects of the same category are acquired (i.e., N sample images with sample classification label information). Further, the server can execute step S302, in which feature extraction is performed on the N sample images using a convolutional neural network (i.e., the initial first convolutional sub-network) to obtain the initial sample features corresponding to the N sample images respectively.

[0237] like Figure 4As shown, the server can execute step S303, which involves obtaining K sample detection object features associated with sample classification label information from N initial sample features. These K sample detection object features are then input into a Graph Neural Network (GNN). Based on the K sample detection object features from the N sample images, the GNN generates sample fusion image features corresponding to each of the N sample images. Specifically, the server can perform object detection on the N sample images using an initial object detection sub-network to obtain K sample detection object features, which are then input into the GNN. The GNN outputs target sample features corresponding to each of the N sample images.

[0238] It can be understood that a graph neural network can include an initial feature embedding subnetwork, an initial attention processing subnetwork, an initial feature enhancement subnetwork, and an initial second convolutional subnetwork. The features of the detected objects from K samples are sequentially passed through the initial feature embedding subnetwork, initial attention processing subnetwork, initial feature enhancement subnetwork, and initial second convolutional subnetwork in the graph neural network. The initial second convolutional subnetwork can then output the target sample features corresponding to N sample images.

[0239] like Figure 4 As shown, the server can execute step S304, in which global pooling is performed on the target sample features through a pooling layer to obtain the category feature vector (i.e., classification feature vector) corresponding to the target sample features. Then, step S305 is executed, in which a loss function is established between the category feature vector and the true category label, and supervised learning is performed using the sample classification label information of N sample images to obtain the target network model. The entire process of steps S301-S305 is end-to-end.

[0240] It should be understood that when the object recognition result is an image detection result, the target network model can be used to perform image detection on the target image; when the object recognition result is a semantic segmentation result, the target network model can be used to perform semantic detection on the target image. Specifically, when the target network model is used for image detection on the target image, both the target network model and its object detection sub-network can be used for object detection (i.e., object detection, image detection). However, the target network model and the object detection sub-network have different detection accuracies. The target network model can utilize the object detection results of the object detection sub-network to determine the image detection result for the target network model, thereby improving the accuracy of image detection.

[0241] It should be understood that the initial network model and the target network model can be collectively referred to as the generalized network model. The initial network model and the target network model are simply names of the generalized network model at different times. During the training phase, the generalized network model can be called the initial network model, and during the prediction phase, it can be called the target network model. Specifically, the initial first convolutional subnetwork in the initial network model and the first convolutional subnetwork in the target network model are the same subnetwork at different times; the initial object detection subnetwork in the initial network model and the object detection subnetwork in the target network model are the same subnetwork at different times; the initial feature embedding subnetwork in the initial network model and the feature embedding subnetwork in the target network model are the same subnetwork at different times; the initial attention processing subnetwork in the initial network model and the attention processing subnetwork in the target network model are the same subnetwork at different times; the initial feature enhancement subnetwork in the initial network model and the feature enhancement subnetwork in the target network model are the same subnetwork at different times; and the initial second convolutional subnetwork in the initial network model and the second convolutional subnetwork in the target network model are the same subnetwork at different times.

[0242] Therefore, this embodiment can perform object detection on multiple (e.g., N) sample images. The features bounding the target box within the initial sample features are used as the features corresponding to the target box (i.e., patch features), and these features are used as nodes. Then, the similarity between different target boxes is learned through the Transformer attention mechanism, and this similarity is used as edge weights to construct a graph network. The node features learned by the graph network are weakly supervised using sample classification label information to achieve object recognition on N sample images. It is understood that after training, the node features exhibit significant spatial attention results; that is, reinforcement learning through the attention mechanism can achieve mutual complementarity between node features. Specifically, when object recognition is semantic segmentation, this embodiment can achieve pixel-level semantic segmentation.

[0243] Further, please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application. The image data processing device 1 may include: a feature acquisition module 11, a matrix generation module 12, a feature enhancement module 13, and an object recognition module 14; further, the image data processing device 1 may also include: a weight generation module 15 and a directed graph construction module 16;

[0244] The feature acquisition module 11 is used to acquire the initial image features corresponding to N target images respectively; all N target images have the same target classification label information; N is a positive integer greater than 1;

[0245] The feature acquisition module 11 includes: an image acquisition unit 111 and a first convolution unit 112;

[0246] Image acquisition unit 111 is used to acquire N target images with the same target classification label information and input the N target images into the target network model; the target network model includes a first convolutional sub-network for performing convolution operations;

[0247] The first convolutional unit 112 is used to perform convolution operations on N target images through the first convolutional sub-network to obtain the initial image features corresponding to the N target images respectively.

[0248] The specific implementation methods of the image acquisition unit 111 and the first convolution unit 112 can be found in the above description. Figure 3 The description of step S102 in the corresponding embodiments will not be repeated here.

[0249] The matrix generation module 12 is used to obtain S embedded object features associated with the target classification label information based on N initial image features, and generate a similarity matrix corresponding to every two embedded object features in the S embedded object features; S is a positive integer greater than or equal to N;

[0250] The target network model also includes an object detection sub-network for object detection, a feature embedding sub-network for feature embedding, and an attention processing sub-network for attention mechanism processing.

[0251] The matrix generation module 12 includes: an object detection unit 121, a feature embedding unit 122, and an attention processing unit 123;

[0252] The object detection unit 121 is used to input N initial image features into the object detection sub-network, and extract S detection object features associated with the target classification label information from the N initial image features through the object detection sub-network; one initial image feature corresponds to one or more detection object features;

[0253] Specifically, the object detection unit 121 is used to input N initial image features into the object detection sub-network. In the object detection sub-network, object detection is performed on N target images based on the N initial image features, and object detection regions for characterizing target objects are determined in the N target images. The target objects have target classification label information. The number of object detection regions is S.

[0254] The object detection unit 121 is specifically used to extract the detection object features corresponding to the S object detection regions.

[0255] The feature embedding unit 122 is used to input S detection object features into the feature embedding sub-network, and to embed the S detection object features through the feature embedding sub-network to obtain S embedded object features with the same feature dimension.

[0256] Attention processing unit 123 is used to input S embedded object features into attention processing subnetwork, and to process the S embedded object features through attention mechanism to generate a similarity matrix corresponding to every two embedded object features in the S embedded object features.

[0257] The attention processing unit 123 is specifically used to input S embedded object features into the attention processing sub-network, and to perform feature operation processing on every two embedded object features among the S embedded object features through the attention processing sub-network to obtain the fused object features corresponding to every two embedded object features.

[0258] Attention processing unit 123 is specifically used to determine the feature size of S embedded object features, and perform feature scaling on the fused object features corresponding to every two embedded object features according to the feature size, so as to obtain the scaled object features corresponding to every two embedded object features.

[0259] Attention processing unit 123 is specifically used to normalize the scaled object features corresponding to each pair of embedded object features, and generate a similarity matrix corresponding to each pair of embedded object features.

[0260] The specific implementation methods of the object detection unit 121, the feature embedding unit 122, and the attention processing unit 123 can be found in the above description. Figure 3 In the corresponding embodiment, steps S103 and Figure 5 The descriptions of steps S1021-S1026 in the corresponding embodiments will not be repeated here.

[0261] Feature enhancement module 13 is used to enhance the features of S embedded object features based on the similarity matrix corresponding to each pair of embedded object features, so as to obtain the enhanced object features corresponding to each embedded object feature in the S embedded object features.

[0262] The object recognition module 14 is used to determine the object recognition results associated with the target classification label information in N target images based on the enhanced object features.

[0263] The target network model also includes a second convolutional subnetwork for performing convolution operations;

[0264] The object recognition module 14 includes: a feature fusion unit 141, a second convolution unit 142, and an object recognition unit 143;

[0265] The feature fusion unit 141 is used to acquire the enhanced object features belonging to the same target image, perform feature fusion on the enhanced object features belonging to the same target image, and obtain the fused image features corresponding to N target images respectively;

[0266] The second convolutional unit 142 is used to input the fused image features into the second convolutional sub-network, and to perform convolution operations on the fused image features through the second convolutional sub-network to obtain the target image features corresponding to the fused image features.

[0267] The object recognition unit 143 is used to determine the object recognition result associated with the target classification label information in N target images based on the target image features.

[0268] The object recognition unit 143 is specifically used to perform upsampling processing on the target image features to obtain the upsampled image features corresponding to the target image features.

[0269] The object recognition unit 143 is specifically used to perform feature adjustment on the upsampled image features according to the image size of N target images to obtain the segmented image features corresponding to the upsampled image features;

[0270] The object recognition unit 143 is specifically used to determine the semantic segmentation results associated with the target classification label information in N target images based on the segmentation image features, and to use the semantic segmentation results as the object recognition results associated with the target classification label information.

[0271] The specific implementation methods of the feature fusion unit 141, the second convolution unit 142, and the object recognition unit 143 can be found in the above description. Figure 3 The description of step S104 in the corresponding embodiment will not be repeated here.

[0272] Optionally, the weight generation module 15 is used to take S embedded object features as nodes and generate edge weights between nodes based on the similarity matrix corresponding to every two embedded object features; one embedded object feature corresponds to one node; one similarity matrix corresponds to one edge weight.

[0273] Directed graph construction module 16 is used to construct a feature-directed graph associated with N target images based on node and edge weights;

[0274] The feature enhancement module 13 is specifically used to enhance the features of S nodes in the directed feature graph based on the edge weights in the directed feature graph, so as to obtain the enhanced object features corresponding to each of the S embedded object features.

[0275] Among them, the S nodes in the directed graph include node S i; i is a positive integer less than or equal to S; the target network model also includes a feature enhancement subnetwork for feature enhancement;

[0276] Feature enhancement module 13 includes: weight determination unit 131, first enhancement unit 132, second enhancement unit 133, and feature aggregation unit 134;

[0277] Weight determination unit 131 is used to determine the weights in the directed feature graph except for node S. i Nodes other than these are designated as auxiliary nodes;

[0278] Weight determination unit 131 is used to obtain auxiliary nodes for node S in the edge weights through the feature enhancement subnetwork. i The relative edge weights, and node S i Self-edge weights; one auxiliary node corresponds to one relative edge weight;

[0279] The first enhancement unit 132 is used for node S i The indicated embedded object features and relative edge weights are used to determine the parameters for node S. i Relative enhancement features for feature enhancement;

[0280] The second enhancement unit 133 is used to acquire node S. i The indicated detection object features are based on node S. i The indicated detection object features and self-edge weights are used to determine the parameters used for node S. i Self-enhancing features that perform feature enhancement;

[0281] Feature aggregation unit 134 is used to aggregate relative enhancement features and self-enhancing features to obtain node S. i The corresponding enhanced object features will be applied to node S. i The corresponding enhanced object features are used as node S i The enhanced object feature corresponding to the indicated embedded object feature.

[0282] The specific implementation methods of the weight determination unit 131, the first enhancement unit 132, the second enhancement unit 133, and the feature aggregation unit 134 can be found in the above description. Figure 6 The descriptions of steps S1031-S1037 in the corresponding embodiments will not be repeated here.

[0283] The specific implementation methods of the feature acquisition module 11, matrix generation module 12, feature enhancement module 13, object recognition module 14, weight generation module 15, and directed graph construction module 16 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S102, Figure 5In the corresponding embodiment, steps S1021-S1026 and Figure 6 The descriptions of steps S1031-S1037 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0284] Further, please see Figure 10 , Figure 10 This is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application. The image data processing device 2 may include: a sample feature acquisition module 21, a sample matrix generation module 22, a sample feature enhancement module 23, and a parameter adjustment module 24;

[0285] The sample feature acquisition module 21 is used to acquire the initial sample features corresponding to N sample images through the initial network model; all N sample images have the same sample classification label information; N is a positive integer greater than 1;

[0286] The sample matrix generation module 22 is used to obtain K sample embedding object features associated with sample classification label information based on N initial sample features in the initial network model, and generate a sample similarity matrix corresponding to every two sample embedding object features in the K sample embedding object features; K is a positive integer greater than or equal to N;

[0287] The sample feature enhancement module 23 is used to enhance the features of K sample embedding object features based on the sample similarity matrix corresponding to each pair of sample embedding object features, so as to obtain the sample enhanced object features corresponding to each sample embedding object feature in the K sample embedding object features.

[0288] The parameter adjustment module 24 is used to adjust the parameters of the initial network model based on the sample augmentation object features and sample classification label information, and use the parameter-adjusted initial network model as the target network model; the target network model is used to perform object recognition on the sample objects indicated by the sample classification label information.

[0289] The initial network model includes a pooling layer for performing pooling operations;

[0290] The parameter adjustment module 24 includes: a convolution operation unit 241, a pooling operation unit 242, a normalization processing unit 243, and a parameter adjustment unit 244.

[0291] The convolution operation unit 241 is used to obtain the sample enhancement object features belonging to the same sample image, perform feature fusion on the sample enhancement object features belonging to the same sample image, and obtain the sample fusion image features corresponding to N sample images respectively.

[0292] Convolution operation unit 241 is used to perform convolution operation on the sample fusion image features to obtain the target sample features corresponding to the sample fusion image features;

[0293] Pooling operation unit 242 is used to input the target sample features into the pooling layer, and perform pooling operation on the target sample features through the pooling layer to obtain the pooled feature vector corresponding to the target sample features;

[0294] The normalization processing unit 243 is used to normalize the pooling feature vector to obtain the classification feature vector corresponding to the pooling feature vector; one sample image corresponds to one classification feature vector.

[0295] The parameter adjustment unit 244 is used to adjust the parameters of the initial network model based on the classification feature vector and sample classification label information corresponding to each of the N sample images, and use the parameter-adjusted initial network model as the target network model.

[0296] The parameter adjustment unit 244 includes: a vector determination subunit 2441, a loss determination subunit 2442, and a parameter adjustment subunit 2443;

[0297] Vector determination subunit 2441 is used to generate a classification label vector for determining the model loss value of the initial network model based on the sample classification label information;

[0298] The loss determination subunit 2442 is used to determine the classification loss value for each sample image based on the classification feature vector and classification label vector corresponding to each of the N sample images.

[0299] The loss determination subunit 2442 is used to obtain the model loss value of the initial network model based on the classification loss value corresponding to each sample image.

[0300] The parameter adjustment subunit 2443 is used to adjust the parameters of the initial network model according to the model loss value. When the parameter-adjusted initial network model meets the model convergence condition, the parameter-adjusted initial network model is used as the target network model.

[0301] The specific implementation methods of the vector determination subunit 2441, the loss determination subunit 2442, and the parameter adjustment subunit 2443 can be found in the above description. Figure 8 The description of step S204 in the corresponding embodiments will not be repeated here.

[0302] The specific implementation methods of the convolution operation unit 241, pooling operation unit 242, normalization processing unit 243, and parameter adjustment unit 244 can be found in the above description. Figure 8The description of step S204 in the corresponding embodiments will not be repeated here.

[0303] The specific implementation methods of the sample feature acquisition module 21, sample matrix generation module 22, sample feature enhancement module 23, and parameter adjustment module 24 can be found in the above description. Figure 8 The descriptions of steps S201-S204 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0304] Further, please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 11 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. Optionally, the network interface 1004 may include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 11 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0305] In such Figure 11 In the computer device 1000 shown, the network interface 1004 provides network communication functions; the user interface 1003 is mainly used to provide an input interface for users; and the processor 1001 can be used to call the device control application stored in the memory 1005.

[0306] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 , Figure 5 , Figure 6 or Figure 8 The description of the image data processing method in the corresponding embodiments can also be performed as described above. Figure 9 The corresponding embodiment describes the image data processing device 1 and Figure 10The description of the image data processing apparatus 2 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0307] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned image data processing apparatus 1 and image data processing apparatus 2. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 3 , Figure 5 , Figure 6 or Figure 8 The description of the image data processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0308] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, causing the computer device to perform the aforementioned actions. Figure 3 , Figure 5 , Figure 6 or Figure 8 The description of the image data processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program products or computer program embodiments related to this application, please refer to the description of the method embodiments of this application.

[0309] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0310] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image data processing method, characterized in that, include: Obtain the initial image features corresponding to N target images; The N target images all have the same target classification label information; N is a positive integer greater than 1. Based on N initial image features, S embedded object features associated with the target classification label information are obtained, and a similarity matrix is ​​generated for every two embedded object features in the S embedded object features; where S is a positive integer greater than or equal to N. Based on the similarity matrix corresponding to each pair of embedded object features, feature enhancement is performed on the S embedded object features to obtain the enhanced object features corresponding to each of the S embedded object features. Based on the enhanced object features, the object recognition results associated with the target classification label information are determined in each of the N target images.

2. The method according to claim 1, characterized in that, The step of obtaining the initial image features corresponding to the N target images includes: N target images with the same target classification label information are obtained, and the N target images are input into a target network model; the target network model includes a first convolutional sub-network for performing convolution operations; The first convolutional sub-network is used to perform convolution operations on the N target images to obtain the initial image features corresponding to the N target images respectively.

3. The method according to claim 2, characterized in that, The target network model further includes an object detection sub-network for object detection, a feature embedding sub-network for feature embedding, and an attention processing sub-network for attention mechanism processing. The step of obtaining S embedded object features associated with the target classification label information based on N initial image features, and generating a similarity matrix corresponding to every two embedded object features in the S embedded object features, includes: N initial image features are input into the object detection sub-network, and the object detection sub-network extracts S detection object features associated with the target classification label information from the N initial image features; one initial image feature corresponds to one or more detection object features; The S detected object features are input into the feature embedding sub-network, and the feature embedding sub-network is used to embed the S detected object features to obtain S embedded object features with the same feature dimension; The S embedded object features are input into the attention processing subnetwork, and the attention processing subnetwork processes the S embedded object features through an attention mechanism to generate a similarity matrix corresponding to every two embedded object features in the S embedded object features.

4. The method according to claim 3, characterized in that, The step of inputting N initial image features into the object detection sub-network and extracting S detection object features associated with the target classification label information from the N initial image features by the object detection sub-network includes: N initial image features are input into the object detection subnetwork. In the object detection subnetwork, object detection is performed on the N target images based on the N initial image features. Object detection regions for characterizing target objects are determined in the N target images. The target objects have the target classification label information. The number of object detection regions is S. Extract the detection object features corresponding to S object detection regions.

5. The method according to claim 3, characterized in that, The step of inputting the S embedded object features into the attention processing subnetwork, and processing the S embedded object features through the attention processing subnetwork using an attention mechanism to generate a similarity matrix corresponding to every two embedded object features in the S embedded object features, includes: The S embedded object features are input into the attention processing sub-network, and the attention processing sub-network performs feature operation processing on every two embedded object features in the S embedded object features to obtain the fused object features corresponding to every two embedded object features. Determine the feature size of the S embedded object features, and perform feature scaling on the fused object features corresponding to every two embedded object features based on the feature size to obtain the scaled object features corresponding to every two embedded object features; The scaling object features corresponding to each pair of embedded object features are normalized to generate a similarity matrix corresponding to each pair of embedded object features.

6. The method according to claim 3, characterized in that, The method further includes: Using the S embedded object features as nodes, and based on the similarity matrix corresponding to every two embedded object features, generate edge weights between the nodes; one embedded object feature corresponds to one node; one similarity matrix corresponds to one edge weight; Based on the nodes and edge weights, a directed feature graph associated with the N target images is constructed; The step of performing feature enhancement on the S embedded object features based on the similarity matrix corresponding to every two embedded object features, to obtain the enhanced object feature corresponding to each of the S embedded object features, includes: Based on the edge weights in the directed feature graph, feature enhancement is performed on S nodes in the directed feature graph to obtain the enhanced object feature corresponding to each of the S embedded object features.

7. The method according to claim 6, characterized in that, The S nodes in the directed graph of the feature include node S. i The i is a positive integer less than or equal to the S; the target network model further includes a feature enhancement subnetwork for feature enhancement; The step of enhancing the features of S nodes in the directed feature graph based on the edge weights in the directed feature graph, to obtain the enhanced object feature corresponding to each of the S embedded object features, includes: Except for node S in the directed graph of the features i Nodes other than these are designated as auxiliary nodes; The auxiliary node for node S is obtained from the edge weights through the feature enhancement subnetwork. i The relative edge weights, and the node S i Self-edge weights; one auxiliary node corresponds to one relative edge weight; Based on the node S i The indicated embedded object features and the relative edge weights are used to determine the parameters for the node S. i Relative enhancement features for feature enhancement; Obtain the node S i The indicated detection object features are based on the node S. i The indicated detection object features and the self-edge weights are used to determine the parameters for the node S. i Self-enhancing features that perform feature enhancement; Feature aggregation is performed on the relative enhancement features and the self-enhancing features to obtain the node S. i The corresponding enhanced object features will be used to enhance the node S. i The corresponding enhanced object features are used as the node S i The enhanced object feature corresponding to the indicated embedded object feature.

8. The method according to claim 2, characterized in that, The target network model also includes a second convolutional subnetwork for performing convolution operations; The step of determining the object recognition result associated with the target classification label information in the N target images based on the enhanced object features includes: Obtain the enhanced object features belonging to the same target image, perform feature fusion on the enhanced object features belonging to the same target image, and obtain the fused image features corresponding to the N target images respectively; The fused image features are input into the second convolutional sub-network, and the second convolutional sub-network performs a convolution operation on the fused image features to obtain the target image features corresponding to the fused image features; Based on the target image features, determine the object recognition results associated with the target classification label information in each of the N target images.

9. The method according to claim 8, characterized in that, The step of determining the object recognition result associated with the target classification label information in each of the N target images based on the target image features includes: The target image features are upsampled to obtain the upsampled image features corresponding to the target image features; Based on the image dimensions of the N target images, the upsampled image features are adjusted to obtain the segmented image features corresponding to the upsampled image features; Based on the segmented image features, semantic segmentation results associated with the target classification label information are determined in the N target images respectively, and the semantic segmentation results are used as object recognition results associated with the target classification label information.

10. An image data processing method, characterized in that, include: Initial sample features are obtained for N sample images using an initial network model; all N sample images have the same sample classification label information; and N is a positive integer greater than 1. In the initial network model, based on N initial sample features, K sample embedding object features associated with the sample classification label information are obtained, and a sample similarity matrix is ​​generated for every two sample embedding object features among the K sample embedding object features; where K is a positive integer greater than or equal to N. Based on the sample similarity matrix corresponding to each pair of sample embedded object features, feature enhancement is performed on the K sample embedded object features to obtain the sample enhanced object features corresponding to each of the K sample embedded object features; Based on the sample augmentation object features and the sample classification label information, the parameters of the initial network model are adjusted, and the parameter-adjusted initial network model is used as the target network model; the target network model is used to perform object recognition on the sample objects indicated by the sample classification label information.

11. The method according to claim 10, characterized in that, The initial network model includes a pooling layer for performing pooling operations; The step of adjusting the parameters of the initial network model based on the sample augmentation object features and the sample classification label information, and using the parameter-adjusted initial network model as the target network model, includes: Obtain the sample enhancement object features belonging to the same sample image, perform feature fusion on the sample enhancement object features belonging to the same sample image, and obtain the sample fused image features corresponding to the N sample images respectively; Perform a convolution operation on the sample fusion image features to obtain the target sample features corresponding to the sample fusion image features; The target sample features are input into the pooling layer, and the pooling layer performs a pooling operation on the target sample features to obtain the pooled feature vector corresponding to the target sample features. The pooling feature vector is normalized to obtain the classification feature vector corresponding to the pooling feature vector; one sample image corresponds to one classification feature vector; Based on the classification feature vector corresponding to each of the N sample images and the sample classification label information, the parameters of the initial network model are adjusted, and the initial network model with adjusted parameters is used as the target network model.

12. The method according to claim 11, characterized in that, The step of adjusting the parameters of the initial network model based on the classification feature vector corresponding to each of the N sample images and the sample classification label information, and using the parameter-adjusted initial network model as the target network model, includes: Based on the sample classification label information, a classification label vector is generated to determine the model loss value of the initial network model; Based on the classification feature vector and the classification label vector corresponding to each of the N sample images, the classification loss value corresponding to each sample image is determined. The model loss value of the initial network model is obtained based on the classification loss value corresponding to each sample image. The parameters of the initial network model are adjusted based on the model loss value. When the initial network model with adjusted parameters meets the model convergence condition, the initial network model with adjusted parameters is used as the target network model.

13. A computer device, characterized in that, include: Processor and memory; The processor is connected to the memory, wherein the memory is used to store a computer program, and the processor is used to invoke the computer program to cause the computer device to perform the method according to any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

15. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium and adapted to be read and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

Images