Training method and device of object classification model, computer device and storage medium

Abstract

The application discloses a kind of training method, device, computer equipment and storage medium of object classification model, the method includes: obtaining source sample field and target sample field;First sample data in source sample field and second sample data in target sample field have the object characteristics of different modalities of training object;Call object classification model to carry out feature calculation between first sample data and the object summary characteristics corresponding to training object, and obtain feature summary deviation;Call object classification model to carry out feature matching to the object summary characteristics corresponding to training object for second sample data, and obtain the predicted object type of the object to which second sample data belongs;According to the predicted object type of the object to which second sample data belongs and the object type label carried by second sample data, generate type prediction deviation;According to feature summary deviation and type prediction deviation, update the model parameter of object classification model.Using the present application, the classification accuracy of the object classification model trained can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method, apparatus, computer device, and storage medium for an object classification model. Background Technology

[0002] With the continuous development of computer networks, machine learning has been applied in many scenarios. Machine learning (ML) is a multidisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0003] In existing applications, when training a model with a small number of samples, features from large samples (with many samples) of different modalities can be transferred to the small samples for training. For example, samples in the small sample can be paired with samples in the large sample, and feature learning can be achieved through the differences in the matching results and labels (such as sample type labels). However, when using this method for model transfer training, because the small sample contains very few samples and the feature differences between different samples are huge, the model trained using this point-to-point method of pairwise matching of samples in the small sample and samples in the large sample is inaccurate. Summary of the Invention

[0004] This application provides a training method, apparatus, computer device, and storage medium for an object classification model, which can improve the classification accuracy of the trained object classification model.

[0005] This application provides a method for training an object classification model, the method comprising:

[0006] Obtain the first sample data from the source sample domain and the second sample data from the target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data from the source sample domain and the second sample data from the target sample domain each have object features of different modalities of the N training objects; both the first sample data from the source sample domain and the second sample data from the target sample domain carry object type labels of their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain;

[0007] The object classification model is invoked to perform feature measurement between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and the feature summary bias for the object summary features is obtained; the model parameters of the object classification model include the object summary features corresponding to the N training objects.

[0008] The object classification model is invoked to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, so as to obtain the predicted object type of the object to which the second sample data in the target sample domain belongs;

[0009] Type prediction bias is generated based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain;

[0010] The model parameters of the object classification model are updated based on the feature summarization bias and type prediction bias to obtain a trained object classification model.

[0011] This application provides a training device for an object classification model, the device comprising:

[0012] The sample acquisition module is used to acquire the first sample data in the source sample domain and the second sample data in the target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each have object features of different modalities of the N training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry object type labels of their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain;

[0013] The feature calculation module is used to call the object classification model to perform feature calculation between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and obtain the feature summary bias for the object summary features; the model parameters of the object classification model include the object summary features corresponding to the N training objects.

[0014] The feature matching module is used to call the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, so as to obtain the predicted object type of the object to which the second sample data in the target sample domain belongs.

[0015] The bias generation module is used to generate type prediction bias based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain.

[0016] The training module is used to update the model parameters of the object classification model based on the feature summary bias and type prediction bias, so as to obtain the trained object classification model.

[0017] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N; the source sample domain contains M first sample data of the k-th training object, where M is a positive integer less than N;

[0018] The process by which the above-mentioned device acquires the object summary features corresponding to each training object in the object classification model includes:

[0019] Select P first sample data points from M first sample data points; P is a positive integer less than M.

[0020] Based on M first sample data and P first sample data, generate P object summary features for the k-th training object.

[0021] Optionally, the above-mentioned device generates P object summary features corresponding to the k-th training object based on P first sample data in the following ways:

[0022] The object classification model is invoked to extract features from the M first sample data, generating sample features corresponding to the M first sample data respectively;

[0023] Generate a first initial sample set to which each of the P first sample data belongs; each first initial sample set contains one of the P first sample data.

[0024] The first sample data that is not included in the P first sample data among the M first sample data is determined as undetermined sample data;

[0025] Based on the feature distances between the sample features of multiple undetermined sample data and the sample features of P first sample data, the multiple undetermined sample data are classified into P first initial sample sets, resulting in P first classified sample sets.

[0026] Generate P summary features of objects based on the P first-classification sample sets.

[0027] Optionally, the above-mentioned device generates P summative features of objects based on P first classification sample sets in the following ways:

[0028] The sample features of the first sample data contained in each first classification sample set are respectively processed by feature averaging to obtain the first average sample features corresponding to each first classification sample set;

[0029] Generate a second initial sample set corresponding to each of the P first average sample features. This second initial sample set is empty.

[0030] Based on the feature distances between the sample features of M first sample data and the features of P first average sample data, the M first sample data are classified into P second initial sample sets, resulting in P second classification sample sets.

[0031] Generate P object summary features based on P sets of second-classification samples.

[0032] Optionally, the process of generating P summary features of objects based on P sets of second-classification samples includes:

[0033] The sample features of the first sample data contained in each second classification sample set are respectively processed by feature averaging to obtain the second average sample features corresponding to each second classification sample set;

[0034] The features of the P second average samples are determined as the summary features of the P objects.

[0035] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N, and the number of object summary features corresponding to a training object is multiple.

[0036] The training module updates the model parameters of the object classification model based on feature summarization bias and type prediction bias to obtain a trained object classification model, including the following methods:

[0037] The object classification model is invoked to obtain the feature differences among the summative features of multiple objects corresponding to the k-th training object; the feature differences are used to maximize the differences among the summative features of multiple objects corresponding to the k-th training object.

[0038] The model parameters of the object classification model are updated based on feature differences, feature summarization bias, and type prediction bias to obtain a trained object classification model.

[0039] Optionally, any second sample data in the target sample domain is represented as the j-th second sample data, where j is a positive integer less than or equal to the total number of second sample data in the target sample domain; the number of object summary features corresponding to a training object is multiple;

[0040] The feature matching module calls the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, thereby obtaining the predicted object type for the second sample data in the target sample domain, including:

[0041] The object classification model is invoked to extract features from the j-th second sample data, generating sample features for the j-th second sample data;

[0042] Feature matching is performed between the sample features of the j-th second sample data and the object summary features corresponding to each training object;

[0043] The object with the smallest feature distance to the sample features of the j-th second sample data is identified as the object type of the training object to which the feature belongs, and thus determined as the predicted object type of the object to which the j-th second sample data belongs.

[0044] Optionally, the bias generation module generates type prediction bias based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain, including:

[0045] Generate type prediction cross-entropy for the j-th second sample data based on the predicted object type of the j-th second sample data and the object type label carried by the j-th second sample data;

[0046] The type prediction loss function is determined based on the type prediction cross-entropy for the j-th second sample data; the type prediction loss function is used to reflect the type prediction bias.

[0047] Optionally, any first sample data in the source sample domain is represented as the i-th first sample data, where i is a positive integer less than or equal to the total number of first sample data in the source sample domain; the number of object summary features corresponding to a training object is multiple;

[0048] The feature calculation module calls the object classification model to perform feature calculations between the first sample data in the source sample domain and the object summary features corresponding to N training objects, and obtains the feature summary bias for the object summary features, including:

[0049] The object classification model is invoked to extract features from the i-th first sample data, generating sample features for the i-th first sample data;

[0050] The training object to which the i-th first sample data belongs is determined as the target training object;

[0051] The summary feature of an object with the largest feature distance from the sample feature of the i-th first sample data among the multiple summary features of objects corresponding to the target training object is determined as the summary feature of the in-class object;

[0052] Among the N types of training objects, the training objects other than the target training object are determined as reference training objects;

[0053] The summary feature of an out-of-class object is determined as the summary feature of an object with the smallest feature distance to the sample feature of the i-th first sample data among the summaries of multiple objects corresponding to the reference training object.

[0054] A summary bias loss function is generated based on the summarized features of objects within the class and the summarized features of objects outside the class; the summary bias loss function is used to reflect the feature summary bias.

[0055] Optionally, the feature calculation module can generate a summary bias loss function based on the summarized features of objects within the class and the summarized features of objects outside the class, including:

[0056] The feature distance between the sample features of the i-th first sample data and the summary features of objects within the class is determined as the first feature distance;

[0057] The feature distance between the sample features of the i-th first sample data and the summary features of out-of-class objects is determined as the second feature distance;

[0058] A summary bias loss function is generated based on the difference between the first feature distance and the second feature distance.

[0059] Optionally, the trained object classification model includes the object summary features corresponding to each trained object.

[0060] The above-mentioned device is also used for:

[0061] Obtain prediction data; the prediction data and the second sample data in the target sample domain have object features of the same modality as the training object;

[0062] The trained object classification model is invoked to extract features from the prediction data, generating prediction features for the prediction data.

[0063] Feature matching is performed between the predicted features and the summaries of the features corresponding to each trained object;

[0064] The object with the smallest feature distance between the matched and predicted features is identified as the training object to which the feature belongs and is then determined as the predicted training object.

[0065] The object type of the training object is determined as the object type of the object to which the prediction data belongs.

[0066] This application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the method of this application.

[0067] This application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described above.

[0068] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0069] This application can obtain first sample data from a source sample domain and second sample data from a target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each possess object features of different modalities of the N training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry object type labels for their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain; an object classification model is invoked to perform classification between the summative object features corresponding to the first sample data in the source sample domain and the N training objects. Feature calculation yields the feature summary bias for the summarized features of the objects. The model parameters of the object classification model include the summarized features corresponding to N training objects. The object classification model is called to perform feature matching between the second sample data in the target sample domain and the summarized features corresponding to each training object, obtaining the predicted object type of the object to which the second sample data belongs in the target sample domain. A type prediction bias is generated based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain. The model parameters of the object classification model are updated based on the feature summary bias and the type prediction bias, resulting in a trained object classification model. Therefore, the method proposed in this application can update the summarized features corresponding to various training objects through the source sample domain, and then match the second sample data with the summarized features corresponding to various training objects. This achieves feature matching between the class template (such as the summarized features) and the second sample data, rather than direct matching between the first and second sample data. This method trains the object classification model, resulting in higher classification accuracy. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0072] Figure 2 This is a schematic diagram illustrating a scenario for predicting object types provided in this application;

[0073] Figure 3 This is a flowchart illustrating a training method for an object classification model provided in this application;

[0074] Figure 4 This is a schematic diagram illustrating a scenario of the correspondence between training samples provided in this application;

[0075] Figure 5 This is a schematic diagram of a model training scenario provided in this application;

[0076] Figure 6 This is a schematic diagram illustrating a scenario provided in this application for predicting the object type of the object to which the predicted data belongs;

[0077] Figure 7 This is a schematic diagram illustrating a model training and application scenario provided in this application;

[0078] Figure 8 This is a schematic diagram of the structure of an object classification model provided in this application;

[0079] Figure 9 This is a flowchart illustrating a method for obtaining summary features of an object provided in this application;

[0080] Figure 10 This is a schematic diagram illustrating a scenario for initializing the summarized characteristics of an object, as provided in this application.

[0081] Figure 11 This is a schematic diagram of the structure of a training device for an object classification model provided in this application;

[0082] Figure 12 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0083] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0084] This application relates to technologies related to artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0085] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0086] This application primarily concerns machine learning within artificial intelligence. Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0087] The machine learning discussed in this application mainly refers to how to perform transfer learning to train an object classification model, and then use the trained object classification model to predict the type of objects. For details, please refer to the following... Figure 3 The description in the corresponding embodiments.

[0088] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, the network architecture may include server 200 and a cluster of terminal devices. The cluster of terminal devices may include one or more terminal devices; the number of terminal devices is not limited here. Figure 1As shown, the multiple terminal devices may specifically include terminal device 100a, terminal device 101a, terminal device 102a, ..., terminal device 103a; as Figure 1 As shown, terminal devices 100a, 101a, 102a, ..., 103a can all connect to server 200 via the network, so that each terminal device can interact with server 200 via the network.

[0089] like Figure 1 The server 200 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart TV, in-vehicle terminal, or other smart terminal. The following description uses the communication between terminal device 100a and server 200 as an example to illustrate the specific implementation of this application.

[0090] Please see also Figure 2 , Figure 2 This is a schematic diagram illustrating a scenario for predicting object types, as provided in this application. For example... Figure 2 As shown, server 200 can train the object classification model to obtain a trained object classification model. The trained object classification model can then predict the object type of the object to which the prediction data provided by terminal device 100a belongs. This process is described below:

[0091] Server 200 can obtain a source sample domain and a target sample domain. The source sample domain contains several first sample data points for N types of training objects, and the target sample domain contains several second sample data points for the same N types of training objects. The first and second sample data points contain object features of different modalities of the N types of training objects. An explanation of the modalities can be found below. Figure 3 The relevant descriptions in the corresponding embodiments are as follows. The target sample domain is the primary sample domain to be trained, while the source sample domain is the sample domain used to assist in training the target sample domain. The model training in this application involves feature transfer learning using sample data and features from the source sample domain and sample features from the target sample domain.

[0092] Server 200 can train the object detection model using the aforementioned source sample domain and target sample domain. This object detection model can include several object summary features corresponding to various training objects to be updated. As the object detection model is trained, the object summary features corresponding to various training objects are continuously updated. Therefore, after training, the completed object detection model will contain updated object summary features corresponding to each training object. These object summary features can be feature vectors. A detailed explanation of the object summary features can be found below. Figure 3 The relevant descriptions in the corresponding embodiments.

[0093] Therefore, the trained object classification model can be applied to the scenario of predicting the object type of the predicted data, as follows:

[0094] The predicted data can be sent from the terminal device 100a to the server 200. After the server 200 obtains the predicted data, it can call the trained object classification model to extract the data features of the predicted data, thereby obtaining the features of the predicted data. These features can be called predicted features, which can be feature vectors.

[0095] Subsequently, the server 200 can perform feature matching between the predicted features and the object summary features corresponding to various training objects in the trained object classification model, and then take the object type of the training object to which the matched object summary feature with the smallest feature distance (such as Euclidean distance) belongs as the object type of the object to which the final predicted data belongs.

[0096] Using the method provided in this application, feature matching between the class template (such as object summary features) of each training object and the prediction data can be achieved, making the object type of the predicted data more accurate.

[0097] Please see Figure 3 , Figure 3 This is a flowchart illustrating a training method for an object classification model provided in this application. The execution entity in this application's embodiments can be a single computer device or a cluster of multiple computer devices. The computer device can be a server or a terminal device. Therefore, the execution entity in this application's embodiments can be a server, a terminal device, or a combination of both. Here, we will use a server as an example to illustrate this. Figure 3 As shown, the method may include:

[0098] Step S101: Obtain the first sample data in the source sample domain and the second sample data in the target sample domain; both the source sample domain and the target sample domain are used to train N kinds of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each have object features of different modalities of N kinds of training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry the object type label of the training object; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain.

[0099] Optionally, the server can obtain a source sample domain and a target sample domain. The source sample domain contains several sample data points, which can be referred to as the first sample data. The number of first sample data points in the source sample domain can be determined according to the actual application scenario. The target sample domain also contains several sample data points, which can be referred to as the second sample data. The number of second sample data points in the target sample domain can be determined according to the actual application scenario.

[0100] In this model training, the target sample domain is the objective for training the model. The source sample domain assists the target sample domain in training the model; for example, features from the first sample data in the source sample domain can be transferred to the features of the second sample data in the target sample domain. The target sample domain can consist of small sample data (i.e., the number of second sample data in the target sample domain is very small), while the source sample domain can consist of large sample data (i.e., the number of first sample data in the source sample domain is very large). The number of second sample data in the target sample domain is far less than the number of first sample data in the source sample domain. Typically, it is difficult to obtain a large amount of second sample data in a short period of time.

[0101] It should be noted that both the source sample domain and the target sample domain are used for training N types of training objects. Specifically, the source sample domain contains the first sample data for each of the N training objects, and the target sample domain contains the second sample data for each of the N training objects. One sample data point (e.g., one first sample data point or one second sample data point) belongs to one training object, and one training object can have several sample data points (e.g., several first sample data points and several second sample data points). Both the first sample data point in the source sample domain and the second sample data point in the target sample domain can carry an object type label for their respective training object. This object type label indicates the object type of the corresponding training object. Therefore, it can be understood that the object type label carried by the first sample data point in the source sample domain (which is the object type label for all N training objects) can be the same as the object type label carried by the second sample data point in the target sample domain (which is also the object type label for all N training objects).

[0102] Furthermore, the first sample data in the source sample domain and the second sample data in the target sample domain each have object features of different modalities of the aforementioned N training objects. Therefore, by adopting this application, cross-modal feature transfer can be achieved, and the different modalities can be determined according to the actual application scenario.

[0103] For example, different modalities can refer to different methods of collecting sample data. N training objects can refer to N kinds of animals. Then, the first sample data in the source sample domain can be the image data of the N kinds of animals taken with a digital camera. The object features under this modality are the image features of the animals taken with the camera. The second sample data in the target sample domain can be the portrait data of the N kinds of animals obtained by hand drawing. The object features under this modality are the image features of the animals obtained by hand drawing.

[0104] For example, the different modalities can refer to different data formats of the sample data, and the N training objects can also refer to N kinds of animals. In this case, the first sample data in the source sample domain can be the audio data of the calls of the N kinds of animals, and the object feature under this modality is the feature of the recorded animal calls. The second sample data in the target sample domain can be the image data of the N kinds of animals that have been photographed, and the object feature under this modality is the image feature of the photographed animals.

[0105] As described above, the different modalities of object features contained in the first sample data in the source sample domain and the second sample data in the target sample domain can refer to: different modalities resulting from different data acquisition methods, such as the first sample data being an image taken with a digital camera, and the second sample data being a hand-drawn image; different modalities resulting from different data formats, such as the first sample data being image data and the second sample data being audio data; or the first sample data being text data and the second sample data being image data; or the first sample data being audio data and the second sample data being text data, etc.

[0106] Due to the different data formats, regardless of whether the sample data is text data, image data, or audio data, the sample data can be represented as a sequence and input into the model for training. For example, for text data, it can be represented as a text sequence (such as the sequence obtained by representing each character in the text data as its corresponding identifier). For image data, it can be represented as an image sequence (such as the sequence obtained by arranging the pixel values ​​at each pixel point). For audio data, it can be represented as an audio sequence (such as the sequence obtained by arranging the individual notes contained in the audio data).

[0107] Please see Figure 4 , Figure 4This is a schematic diagram illustrating a scenario of the correspondence between training samples provided in this application. For example... Figure 4 As shown, the N training subjects can be N kinds of animals, including animals such as tigers, rabbits, cats, and dogs. Therefore, the first sample data contained in the source sample domain can be images of the N kinds of animals taken with a digital camera, and the second sample data contained in the target sample domain can be hand-drawn images of the N kinds of animals obtained by hand.

[0108] Specifically, such as Figure 4 As shown, the first sample data of the training object "tiger" in the source sample domain can be a photographed image of a tiger, which carries an object type label for the tiger. The object type label is used to indicate that the object to which the photographed image belongs is of the tiger type. The second sample data of the training object "tiger" in the target sample domain can be a hand-drawn image of a tiger, which also carries an object type label for the tiger. The object type label is used to indicate that the object to which the hand-drawn image belongs is of the tiger type.

[0109] Similarly, the first sample data of the training object "rabbit" in the source sample domain can be a photographed image of a rabbit, which carries an object type label for rabbit, indicating that the object to which the photographed image belongs is of the type rabbit; the second sample data of the training object "rabbit" in the target sample domain can be a hand-drawn image of a rabbit, which also carries an object type label for rabbit, indicating that the object to which the hand-drawn image belongs is of the type rabbit.

[0110] Similarly, the first sample data of the training object "cat" in the source sample domain can be a photographed image of a cat, which carries a cat object type label, and the object type label is used to indicate that the object to which the photographed image belongs is of the cat type; the second sample data of the training object "cat" in the target sample domain can be a hand-drawn image of a cat, which also carries a cat object type label, and the object type label is used to indicate that the object to which the hand-drawn image belongs is of the cat type.

[0111] Similarly, the first sample data of the training object "dog" in the source sample domain can be a photographed image of a dog, which carries a dog object type label, and the object type label is used to indicate that the object to which the photographed image belongs is of the dog type; the second sample data of the training object "dog" in the target sample domain can be a hand-drawn image of a dog, which also carries a dog object type label, and the object type label is used to indicate that the object to which the hand-drawn image belongs is of the dog type.

[0112] Step S102: The object classification model is invoked to perform feature measurement between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and the feature summary bias for the object summary features is obtained; the model parameters of the object classification model include the object summary features corresponding to the N training objects.

[0113] Optionally, the model parameters of the object classification model can include object summary features corresponding to each of the N training objects. The number of object summary features corresponding to each training object can be the same or different. For example, each training object can correspond to P object summary features, where P is a positive integer, and the specific value of P is determined according to the actual application scenario. The number of object summary features corresponding to each training object is determined according to the actual application scenario and is not limited thereto.

[0114] In this context, it can be understood that one or more object summary features (usually multiple) corresponding to a training object are one or more class features (which can be denoted as class templates) extracted from that training object. During the training of the object classification model, the object summary features corresponding to each training object will also be continuously updated iteratively.

[0115] Optionally, object summary features are essentially feature vectors. An object summary feature is a feature vector. When initially training the object classification model, the model can contain initial object summary features corresponding to various training objects. These initial object summary features can be pre-defined, such as feature vectors containing zero elements. Alternatively, these initial object summary features can be generated from the source sample domain using the k-means algorithm (a clustering algorithm based on Euclidean distance). The specific process of generating the initial object summary features corresponding to various training objects using the k-means algorithm from the first sample data in the source sample domain can be found below. Figure 9 The description in the corresponding embodiment.

[0116] The server can call the object classification model to perform feature calculations between the first sample data in the source sample domain and the object summary features corresponding to N training objects, so as to calculate the feature summary bias for the object summary features. This feature summary bias is used to characterize the accuracy of the object summary features corresponding to the various training objects that have been updated.

[0117] Optionally, the server can perform feature calculation by measuring the difference between the maximum feature distance within a class and the minimum feature distance outside the class, and then using this difference to obtain the feature summary bias. The specific process is described below:

[0118] Any first sample data in the source sample domain can be represented as the i-th first sample data, where i is a positive integer less than or equal to the total number of first sample data in the source sample domain.

[0119] The server can invoke an object classification model to generate sample features for the i-th first sample data. The object classification model may include a feature extractor, which the server can use to extract features from the i-th first sample data, generating sample features that can be a feature vector.

[0120] It is understandable that since model training involves multiple iterations, if sample features of a certain sample data (such as a first sample data or a second sample data) have already been generated during the training of an object classification model, then if the sample features of that sample data need to be used multiple times during that training process, the sample features of that sample data already generated during that training process can be used directly (i.e. reused) without having to generate them again.

[0121] The training object to which the i-th first sample data belongs can be called the target training object. Therefore, the object summary feature with the largest feature distance among the multiple object summary features corresponding to the target training object and the sample features of the i-th first sample data can be taken as the in-class object summary feature.

[0122] Since the summary features of the object are feature vectors, and the sample features of the i-th first sample data are also feature vectors, it can be understood that the feature distance between the summary features of the object and the sample features of the i-th first sample data is the vector distance, which can be Euclidean distance.

[0123] Correspondingly, the training objects other than the target training objects mentioned above among the N types of training objects can be called reference training objects. The object summary feature with the smallest feature distance to the sample feature of the i-th first sample data among the multiple object summary features corresponding to the reference training object can be used as the out-of-class object summary feature.

[0124] Furthermore, the server can generate a summary bias loss function based on the aforementioned summary features of in-class and out-of-class objects. This summary bias loss function reflects the feature summarization bias for object summary features. The larger the summary bias loss function, the greater the feature summarization bias, and the less accurate the object summary features corresponding to the current target training object. This summary bias loss function ensures that the maximum feature distance (e.g., the feature distance between the summary features of in-class objects and the sample features of the i-th first sample data) within the same class (such as the target training object class) is less than the minimum distance (e.g., the feature distance between the summary features of out-of-class objects and the sample features of the i-th first sample data) between different classes (such as the target training object class and non-target training object classes). In other words, it guarantees that the maximum distance between the summary features of the same training object and the sample features of the first sample data is also less than the minimum distance between the summary features of different training objects and the sample features of the first sample data.

[0125] Here, the i-th first sample data can be denoted as x. t, Let t represent the source sample domain. The feature distance between the sample features of the i-th first sample data and the summative features of the objects within the class can be called the first feature distance, which can be denoted as w. + The feature distance between the sample features of the i-th first sample data and the summary features of out-of-class objects can be called the second feature distance, which can be denoted as w. - Therefore, as shown in Equations (1), (2), and (3) below, the summation bias loss function corresponding to the i-th first sample data is loss. i It could be:

[0126] loss i =w + -w - <δ (1)

[0127] w + =argmax d(f(x) t,i ),z m (2)

[0128] w - =argmin d(f(x) t,i ),z m‘ (3)

[0129] Here, δ can be a value approaching 0, summarizing the bias loss function. i The first feature distance w + Distance w to the second feature - The difference between them, z mIt can be any one of the multiple object summary features corresponding to the target training object, and the value of m can be 1 to the total number of multiple object summary features corresponding to the target training object. t,i ) is the i-th first sample data x t, The sample features, therefore, argmax d(f(x) t,i ),z m The feature distance (i.e., the maximum feature distance within the class) is the distance between the features of the i-th first sample data and the summation features of the objects within the class. m‘ It can be any one of the multiple object summary features corresponding to the reference training object, and the value of m' can be 1 to the total number of multiple object summary features corresponding to the reference training object. argmin d(f(x) t,i ),z m‘ The feature distance (i.e., the minimum feature distance between the sample features of the i-th first sample data and the summary features of the out-of-class objects) is the feature distance between the sample features of the i-th first sample data and the summary features of the out-of-class objects.

[0130] If a training process for an object classification model involves multiple first sample data points (which can include multiple first sample data points corresponding to various training objects), then the summation bias loss functions corresponding to each of these multiple first sample data points can be summed to obtain the final summation bias loss function for that training process. The process of obtaining the summation bias loss function corresponding to each first sample data point is the same as the process of obtaining the summation bias loss function corresponding to the i-th first sample data point described above.

[0131] Step S103: Call the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, and obtain the predicted object type of the object to which the second sample data in the target sample domain belongs.

[0132] Optionally, the server can invoke the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, thereby obtaining the predicted object type of the object to which the second sample data in the target sample domain belongs. This object type can be referred to as the predicted object type. During one training process of the model, one second sample data can correspond to one predicted object type.

[0133] It should be noted that the object classification model is trained iteratively, with each subsequent training iteration building upon the previous one. Since the principle of each training iteration is the same, this application uses the process of training the model once as an example for illustration.

[0134] Any second sample data in the target sample domain can be represented as the j-th second sample data, where j is a positive integer less than or equal to the total number of second sample data in the target sample domain. The feature extractor in the object classification model can generate the sample features of the j-th second sample data. Then, the server can perform feature matching between the sample features of the j-th second sample data and several object summary features corresponding to each training object. The object type of the training object to which the object summary feature (which can be Euclidean distance) with the smallest feature distance (which is the smallest) between the matched sample features (which are the feature vectors) of the j-th second sample data belongs can be used as the predicted object type of the object to which the j-th second sample data belongs (i.e., the training object to which it belongs).

[0135] Step S104: Generate type prediction bias based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain.

[0136] Optionally, since the object type label carried by the second sample data indicates the actual object type of the object to which the second sample data belongs, the server can generate a type prediction bias based on the deviation between the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data. This type prediction bias reflects the accuracy of the current object classification model in predicting the object type. The larger the type prediction bias, the lower the accuracy of the current object classification model in predicting the object type; conversely, the smaller the type prediction bias, the higher the accuracy of the current object classification model in predicting the object type.

[0137] Specifically, the j-th second sample data mentioned above can be represented as x. s,j As shown in formula (4) below, the predicted object type of the j-th second sample data can be represented as:

[0138] y(min d(f(x s,j ),w1)) (4)

[0139] Where, f(x) s,j ) represents the sample feature of the j-th second sample data, w l It can be any summative feature among all the summative features corresponding to all training objects, and l can be a positive integer less than or equal to the total number of summative features corresponding to all training objects. Therefore, min d(f(x) s,j ),w l ) is the minimum feature distance among the feature distances between the summarized features of each training object and the sample features of the j-th second sample data, y(min d(f(x)). s,j ),wl The predicted minimum feature distance corresponds to the object type of the training object whose summarized features belong to this type of object. This y(min d(f(x)) represents the object type of the training object. s,j ),w l This can be represented as a vector where only the position of the predicted object type for the j-th second sample data has a value of 1, while the positions of other object types in the vector have values ​​of 0. The dimensions of the sample features and the summative features of the object data can be the same.

[0140] Here, the actual object type indicated by the object type label carried by the j-th second sample data can be represented as y. s,j , that y s,j It can also be represented as a vector in which only the position of the actual object type of the j-th second sample data has a value of 1, and the positions of other object types in the vector have values ​​of 0.

[0141] Therefore, the predicted object type y(min d(f(x)) is obtained through the j-th second sample data. s,j ),w l The actual object type is represented as y. s,j This generates the type prediction loss function corresponding to the j-th second sample data. This type prediction loss function reflects the type prediction bias of the current object classification model in predicting the object type of the j-th second sample data. The larger the type prediction loss function, the larger the type prediction bias, and vice versa.

[0142] This type of prediction loss function can be represented as loss j As shown in the following formula (5), this loss j It could be:

[0143] loss j =y s,j log y(min d(f(x s,j ),w l (5)

[0144] As shown in the above formula (5), it can be seen that the type prediction loss function is the cross-entropy loss between the actual object type and the predicted object type (which can be called the type prediction cross-entropy for the j-th second sample data).

[0145] If a training process for an object classification model involves multiple second sample data sets (which can include multiple second sample data sets corresponding to various training objects), then the type prediction loss functions corresponding to each of these multiple second sample data sets can be summed to obtain the final type prediction loss function for that training process. The process of obtaining the type prediction loss function for each second sample data set is the same as the process of obtaining the summary bias loss function corresponding to the j-th second sample data set.

[0146] Step S105: Update the model parameters of the object classification model based on the feature summary bias and type prediction bias to obtain the trained object classification model.

[0147] Optionally, any one of the N training objects mentioned above can be represented as the k-th training object, where k is a positive integer less than or equal to N. The server can also call the object classification model to obtain the feature differences between the summary features of multiple objects corresponding to the k-th training object. These feature differences are used to maximize the feature differences between the summary features of multiple objects corresponding to the k-th training object, ensuring significant differences between the summary features of multiple objects corresponding to the same training object (e.g., the k-th training object), thereby reducing redundancy among the summary features of multiple objects corresponding to the same training object.

[0148] Optionally, the server can call the object classification model to generate a difference loss function among the summarized features of multiple objects corresponding to the k-th training object. This difference loss function is used to reflect the feature differences among the summarized features of multiple objects corresponding to the k-th training object, and can be expressed as loss. k As shown in the following formula (6), the difference loss function is... k It could be:

[0149] loss k =max∑ k ‖t1-t2‖ (6)

[0150] Where t1 and t2 can be any two object summary features from multiple object summary features corresponding to the k-th training object, and t1 and t2 can be different object summary features. ‖t1-t2‖ represents the L-value between object summary feature t1 and object summary feature t2. 2,1 Norm.

[0151] Since the object classification model is trained using N types of training objects, the difference loss functions corresponding to each type of training object can be summed to obtain the final difference loss function during the training process of the object classification model. The process of obtaining the difference loss function for each type of training object is the same as the process of obtaining the difference loss function for the k-th type of training object described above.

[0152] Therefore, the server can update the model parameters of the object classification model using the summarization bias loss function for object summary features, the type prediction loss function for object type, and the difference loss function between summary features of similar objects, all obtained as described above. By iteratively updating the model parameters of the object classification model using the above process, a trained object classification model can be obtained. This trained model includes the finally updated object summary features corresponding to each training object, as well as the feature extractor with updated model parameters. The model parameters of the object classification model can include the object summary features corresponding to each training object and the parameters of the feature extractor.

[0153] The goal of updating the model parameters of the object classification model by using the summary bias loss function for summarizing object features, the type prediction loss function for object types, and the difference loss function for summarizing features of similar objects can be: to update the model parameters of the object classification model so that the summary bias loss function and the type prediction loss function are minimized, and the difference loss function is maximized.

[0154] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating a model training scenario provided in this application. The server can input the first sample data from the source sample domain and the second sample data from the target sample domain into the object detection model. The object detection model can then generate feature summary bias for object summary features, type prediction bias for object types, and feature differences between different object summary labels for the same training object. By backpropagating these feature differences, type prediction bias, and feature summary bias to the object detection model, the model parameters of the object detection model can be updated using these features. Finally, the object classification model with updated model parameters can be used as the trained object classification model.

[0155] Furthermore, a well-trained object classification model can be used to predict the object type of the data, where the predicted data and the second sample data share the same modalities of the training objects. For example, if the N training objects are N kinds of animals, and the second sample data in the target sample domain is a hand-drawn image of any one of the N animals, then the predicted data can also be a hand-drawn image of any one of the N animals.

[0156] Specifically, the server can obtain the prediction data and call the trained object classification model (such as calling the feature extractor in the trained object classification model) to extract features from the prediction data to generate the feature vector of the prediction data. The feature vector of the prediction data can be called the prediction feature.

[0157] Furthermore, the server can perform feature matching between the predicted features and the object summary features corresponding to each trained object. The training object to which the matched object summary feature has the smallest feature distance (such as Euclidean distance) with the predicted features belongs can be called the predicted training object, and the object type of the predicted training object can be used as the object type of the determined predicted data.

[0158] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating a scenario provided in this application for predicting the object type of the object to which the predicted data belongs. For example... Figure 6 As shown, the above N types of training objects can include training object 1, training object 11 and training object 111. The object type of training object 1 is object type 1, the object type of training object 11 is object type 11, and the object type of training object 111 is object type 111.

[0159] In the trained object classification model, the object summary features corresponding to training object 1 include object summary feature 1, object summary feature 2 and object summary feature 3; the object summary features corresponding to training object 11 include object summary feature 11, object summary feature 22 and object summary feature 33; and the object summary features corresponding to training object 111 include object summary feature 111, object summary feature 222 and object summary feature 333.

[0160] Therefore, the server can call the trained object classification model to extract features from the prediction data and obtain the prediction features of the prediction data. Then, the server can perform feature matching between the prediction features and object summary features 1, 2, 3, 11, 22, 33, 111, 222, and 333 respectively. Finally, if the feature distance between object summary feature 11 and the prediction feature is the smallest, then the object type 11 of the training object 11 to which object summary feature 11 belongs can be taken as the object type of the object to which the prediction data belongs in the final prediction.

[0161] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating a model training and application scenario provided in this application. For example... Figure 7As shown, the server can first create a dataset, which includes a source domain and a target domain. The source domain is the aforementioned source sample domain, and the target domain is the aforementioned target sample domain.

[0162] Furthermore, the server can perform offline model training using the established source and target domains, i.e., offline training of the object classification model. Here, the INPUT in the offline model training process can be the sample data in the source and target domains, and the OUTPUT can be the predicted object type of the input sample data.

[0163] After the object detection model is trained, online model inference can be performed using the trained object detection model. In the online model inference process, the INPUT can be the sample data in the target domain or the predicted data of the object features with the same modality as the second sample data. The trained object detection model can contain the object summary features corresponding to various training objects. The OUTPUT can be obtained through the object summary features. The OUTPUT can predict the object type of the sample data in the input target sample domain or the object to which the predicted data belongs.

[0164] The object classification model in this application can be any network.

[0165] Please see Figure 8 , Figure 8 This is a structural diagram of an object classification model provided in this application. For example... Figure 8 As shown, the object classification model can be a ResNet-18 network or other networks; there are no restrictions on this.

[0166] Here, Class1 to ClassN represent N types of training objects, and each type of training object can correspond to P objects summarizing features. For example... Figure 8As shown, the ResNet-18 network can specifically include a "7x7 conv,32" network layer (i.e., a convolutional layer with a size of 7*7 and 32 channels), a "BN,ReLU" network layer (i.e., a normalization layer and an activation layer), a "Max Pooling" network layer (i.e., a maximum pooling layer), two "ResBlock,64" network layers (i.e., network blocks of the ResNet network with 64 channels), two "ResBlock,128" network layers (i.e., network blocks of the ResNet network with 128 channels), two "ResBlock,256" network layers (i.e., network blocks of the ResNet network with 256 channels), two "ResBlock,512" network layers (i.e., network blocks of the ResNet network with 512 channels), and an "Average Pooling" network layer (i.e., an average pooling layer).

[0167] Using the method provided in this application, when transferring the features of sample data in the source sample domain, it is not necessary to perform pairwise matching between the sample features of the sample data in the source sample domain and the sample features of the sample data in the target sample domain. Instead, it is only necessary to perform feature matching between the sample features of the sample data in the target sample domain and the summaries of several objects corresponding to various training objects. Since the number of summaries of objects corresponding to all training objects is far less than the first sample data of all training objects, feature matching between the sample features of the sample data in the target sample domain and the summaries of objects corresponding to various training objects can greatly reduce the matching time, thereby reducing the model training time, training difficulty, and training cost.

[0168] Furthermore, since this application summarizes and extracts class features of various training objects to obtain object summary features that represent class features of various training objects, and then matches the object summary features with the sample features of the second sample data, it realizes feature matching between class template (i.e., object summary features) and point (i.e., second sample data). Compared with the point (i.e., first sample data) to point (i.e. second sample data) feature matching method used in existing applications, the matching results (such as the matched object type) of the feature matching between class template (i.e., object summary features) and point (i.e. second sample data) provided by this application are more accurate, and solve the problem of inaccurate point-to-point matching caused by the huge differences between different samples (such as different second sample data) in the case of small samples (such as the target sample domain).

[0169] The following describes the experimental results (including prediction accuracy for object types in various sample domains) of several model training methods, including CCSA, Supervised, SGA, DSNE, and AdaE, using ResNet-18 (a residual network) as the backbone network and a feature dimension (such as the dimension of object summary features and sample features) of 512. As shown in Table 1 below, the experimental results include:

[0170] Model training methods A→D D→A W→D D→W A→W W→A CCSA 0.8864 0.6626 0.9926 0.9459 0.8774 0.6590 Supervised 0.8938 0.6439 0.9827 0.9516 0.8789 0.6600 SGA 0.7802 0.5033 0.8198 0.7948 0.7806 0.5015 DSNE 0.8790 0.6516 0.9778 0.9729 0.8675 0.6824 AdaE 0.9086 0.6909 0.9654 0.9643 0.8889 0.6905

[0171] Table 1

[0172] In Table 1, A represents Amazon, which is a collection of images downloaded from Amazon; D represents Digits, which is a collection of hand-drawn images; and W represents Webcam, which is a collection of images taken with a digital camera. CCSA and DSNE are representative methods of few-shot transfer learning, respectively. The Supervised method refers to the method of pre-training the model in the source sample domain and then fine-tuning the parameters in the target sample domain. SGA refers to the method of learning the model only through the target sample domain. AdaE is the method used in this application. As can be seen from the experimental results in Table 1 above, the method provided in this application has the highest prediction accuracy for object types in most types of source and target sample domains. In Table 1 above, the part before the arrow "→" represents the source sample domain (such as A, D, or W), and the part after the arrow "→" represents the target sample domain (such as A, D, or W).

[0173] This application can obtain first sample data from a source sample domain and second sample data from a target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each possess object features of different modalities of the N training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry object type labels for their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain; an object classification model is invoked to perform classification between the summative object features corresponding to the first sample data in the source sample domain and the N training objects. Feature calculation yields the feature summary bias for the summarized features of the objects. The model parameters of the object classification model include the summarized features corresponding to N training objects. The object classification model is called to perform feature matching between the second sample data in the target sample domain and the summarized features corresponding to each training object, obtaining the predicted object type of the object to which the second sample data belongs in the target sample domain. A type prediction bias is generated based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain. The model parameters of the object classification model are updated based on the feature summary bias and the type prediction bias, resulting in a trained object classification model. Therefore, the method proposed in this application can update the summarized features corresponding to various training objects through the source sample domain, and then match the second sample data with the summarized features corresponding to various training objects. This achieves feature matching between the class template (such as the summarized features) and the second sample data, rather than direct matching between the first and second sample data. This method trains the object classification model, resulting in higher classification accuracy.

[0174] Please see Figure 9 , Figure 9 This is a flowchart illustrating a method for obtaining summary features of an object provided in this application. The execution entity in the embodiments of this application can be the same as described above. Figure 3 The execution entities in the corresponding embodiments are the same, such as servers. This application describes the process of obtaining the object summary features corresponding to each initial training object when starting to train the object classification model. The content described in this application can be the same as the above. Figure 3 Combined with the content described in the corresponding embodiments. For example Figure 9 As shown, the method may include:

[0175] Step S201: Select P first sample data from M first sample data.

[0176] Optionally, any one of the N training objects can be represented as the k-th training object, where k is a positive integer less than or equal to N. The source sample domain can contain M first sample data of the k-th training object, and each of the M first sample data carries the object type label of the k-th training object.

[0177] First, when preparing to train the object classification model, we can determine the number of object summary features needed to generate for the k-th training object. This number can be P, indicating that P object summary features need to be generated for the k-th training object. The value of P can be determined based on the actual application scenario and is not restricted. For example, for training objects with complex features, the value of P can be larger, while for training objects with simpler features, the value of P can be smaller. Therefore, the server can first randomly select P first sample data points from the M first sample data points in the source sample domain.

[0178] Step S202: Call the object classification model to generate sample features corresponding to M first sample data respectively.

[0179] Optionally, the server can call the current object classification model (such as calling the feature extractor of the object classification model) to extract features from the M first sample data, thereby generating sample features corresponding to each of the M first sample data. Each first sample data can correspond to one sample feature, which can be a feature vector.

[0180] The initial sample features corresponding to the M first sample data can also be generated in other ways, such as by a pre-trained feature extractor, without any restrictions.

[0181] Step S203: Generate a first initial sample set to which each of the P first sample data belongs; a first initial sample set contains one of the P first sample data.

[0182] Optionally, the server can generate P first initial sample sets to which each first sample data belongs. Each first sample data corresponds to a first initial sample set, and the initial sample set corresponding to each first sample data contains that first sample data.

[0183] Step S204: Determine the first sample data other than P first sample data from the M first sample data as undetermined sample data.

[0184] Optionally, all of the first sample data except for P first sample data can be regarded as undetermined sample data.

[0185] Step S205: Based on the feature distances between the sample features of multiple undetermined sample data and the sample features of P first sample data, classify the multiple undetermined sample data into P first initial sample sets to obtain P first classification sample sets.

[0186] Optionally, the server can classify several undetermined sample data into P initial sample sets. The specific process can be as follows:

[0187] The server can calculate the feature distance between the sample features of each undetermined sample data and the sample features of P first sample data, that is, calculate the feature distance pairwise. Then, the server can add each undetermined sample data to the first initial sample set to which the first sample data with the minimum feature distance between each undetermined sample data belongs.

[0188] For example, if the feature distance (such as Euclidean distance) between the sample features of the undetermined sample data 1 and the sample features of the first sample data 2 among the M first sample data is the smallest, then the undetermined sample data 1 can be added to the first initial sample set to which the first sample data 2 belongs.

[0189] After all the undetermined sample data are categorized and added to their corresponding first initial sample sets, each of these first initial sample sets can be called a first categorized sample set, resulting in P first categorized sample sets. One initial sample set corresponds to one first categorized sample set.

[0190] Step S206: Generate P object summary features based on the P first classification sample sets.

[0191] Optionally, the server can perform feature averaging (i.e., averaging) on ​​the sample features of several first sample data contained in each first classification sample set, and then obtain the average sample features corresponding to each first classification sample set. The average sample features can be called the first average sample features. One first classification sample set corresponds to one first average sample feature, and there are a total of P first average sample features.

[0192] For example, if the first classification sample set 1 contains first sample data 1, first sample data 2, and first sample data 3, and the dimensions of each sample feature are the same, then the server can perform feature averaging on the sample features of first sample data 1, first sample data 2, and first sample data 3 to obtain the first average sample feature corresponding to the first classification sample set 1. This first average sample feature can also be a feature vector, with the same dimensions as each sample feature. Each vector element in the first average sample feature can be the average of the vector elements at corresponding positions in the sample features of first sample data 1, first sample data 2, and first sample data 3. A vector element of the feature vector can be a number (i.e., an element) within that feature vector.

[0193] Furthermore, the server can generate a second initial sample set corresponding to each of the first average sample features. This second initial sample set can be empty and does not contain any sample data.

[0194] Therefore, similarly, the server can classify the M first sample data into the second initial sample set corresponding to each first average sample feature. This process can be: the server can add the M first sample data to the second initial sample set corresponding to the first average sample feature that has the minimum feature distance to the sample features of the M first sample data.

[0195] For example, if the feature distance (such as Euclidean distance) between the sample feature of the first sample data 1 and the first average sample feature 1 among the M first sample data is the smallest, then the first sample data 1 can be added to the second initial sample set corresponding to the first average sample feature.

[0196] After all M first sample data are classified into their respective second initial sample sets, each of these second initial sample sets can be called a second classification sample set, resulting in a total of P second classification sample sets.

[0197] Similarly, the server can perform feature averaging (i.e., averaging) on ​​the sample features of several first sample data contained in each second classification sample set, thereby obtaining the average sample features corresponding to each second classification sample set. These average sample features can be called second average sample features. One second classification sample set corresponds to one second average sample feature, and there are a total of P second average sample features.

[0198] Therefore, the server can directly use the P second average sample features as the P object summary features corresponding to the k-th training object. Alternatively, the server can also use the P second average sample features to reclassify the M first sample data, obtaining a new set of P classified samples. This process is the same as classifying the M first sample data using the P first average sample features. The server can continuously iterate and classify the M first sample data in this way (e.g., classifying the M first sample data using the P first average sample features) (the number of iterations can be set according to the actual application scenario) to obtain the final set of P classified samples. Then, the server calculates the average sample features corresponding to the final set of P classified samples, and uses the average sample features corresponding to the final set of P classified samples as the final P object summary features corresponding to the k-th training object.

[0199] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating a scenario for initializing the summarized characteristics of an object, as provided in this application. For example... Figure 10 As shown, firstly, P first sample data can be selected from the M first sample data of the k-th training object. Then, each undetermined sample data can be classified into the sample set corresponding to the P first sample data respectively, thus obtaining P first classification sample sets.

[0200] Subsequently, the server can calculate the first average sample feature corresponding to each first classification sample set, obtaining a total of P first average sample features. Then, the server can use these P first average sample features to classify M first sample data, obtaining P second classification sample sets.

[0201] Similarly, the server can calculate the second average sample features corresponding to each second classification sample set, resulting in a total of P second average sample features. Then, the server can use these P second average sample features to reclassify the M first sample data, obtaining P reclassified classification sample sets. This process is repeated iteratively to obtain the final P classification sample sets for the M first sample data. Finally, the average sample features corresponding to each of these P classification sample sets are used as the object summary features for the k-th training object, which corresponds to a total of P object summary features.

[0202] This application generates initial object summary features corresponding to each training object by using M first sample data of the k-th training object in the source sample domain. This allows the model to be iteratively updated based on the initial object summary features corresponding to each training object during model training, resulting in more accurate updated object summary features corresponding to each training object.

[0203] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a training device for an object classification model provided in this application. The training device for the object classification model can be a computer program (including program code) running on a computer device. For example, the training device for the object classification model can be an application software, and can be used to execute the corresponding steps in the method provided in the embodiments of this application. Figure 11 As shown, the training device 1 for the object classification model may include: a sample acquisition module 11, a feature calculation module 12, a feature matching module 13, a bias generation module 14, and a training module 15.

[0204] The sample acquisition module 11 is used to acquire first sample data in the source sample domain and second sample data in the target sample domain; both the source sample domain and the target sample domain are used to train N kinds of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each have object features of different modalities of N kinds of training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry object type labels of their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain;

[0205] The feature calculation module 12 is used to call the object classification model to perform feature calculation between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and obtain the feature summary bias for the object summary features; the model parameters of the object classification model include the object summary features corresponding to the N training objects.

[0206] Feature matching module 13 is used to call the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, so as to obtain the predicted object type of the object to which the second sample data in the target sample domain belongs;

[0207] The deviation generation module 14 is used to generate a type prediction deviation based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain.

[0208] Training module 15 is used to update the model parameters of the object classification model based on feature summary bias and type prediction bias, so as to obtain the trained object classification model.

[0209] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N; the source sample domain contains M first sample data of the k-th training object, where M is a positive integer less than N;

[0210] The process by which the above-mentioned device 1 acquires the object summary features corresponding to each training object in the object classification model includes:

[0211] Select P first sample data points from M first sample data points; P is a positive integer less than M.

[0212] Based on M first sample data and P first sample data, generate P object summary features for the k-th training object.

[0213] Optionally, the method by which the above-mentioned device 1 generates the summative features of P objects corresponding to the k-th training object based on P first sample data includes:

[0214] The object classification model is invoked to extract features from the M first sample data, generating sample features corresponding to the M first sample data respectively;

[0215] Generate a first initial sample set to which each of the P first sample data belongs; each first initial sample set contains one of the P first sample data.

[0216] The first sample data that is not included in the P first sample data among the M first sample data is determined as undetermined sample data;

[0217] Based on the feature distances between the sample features of multiple undetermined sample data and the sample features of P first sample data, the multiple undetermined sample data are classified into P first initial sample sets, resulting in P first classified sample sets.

[0218] Generate P summary features of objects based on the P first-classification sample sets.

[0219] Optionally, the method by which the above-mentioned device 1 generates P object summary features based on P first classification sample sets includes:

[0220] The sample features of the first sample data contained in each first classification sample set are respectively processed by feature averaging to obtain the first average sample features corresponding to each first classification sample set;

[0221] Generate a second initial sample set corresponding to each of the P first average sample features. This second initial sample set is empty.

[0222] Based on the feature distances between the sample features of M first sample data and the features of P first average sample data, the M first sample data are classified into P second initial sample sets, resulting in P second classification sample sets.

[0223] Generate P object summary features based on P sets of second-classification samples.

[0224] Optionally, the process of generating P summary features of objects based on P sets of second-classification samples in the above-mentioned device 1 includes:

[0225] The sample features of the first sample data contained in each second classification sample set are respectively processed by feature averaging to obtain the second average sample features corresponding to each second classification sample set;

[0226] The features of the P second average samples are determined as the summary features of the P objects.

[0227] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N, and the number of object summary features corresponding to a training object is multiple.

[0228] Training module 15 updates the model parameters of the object classification model based on feature summarization bias and type prediction bias, obtaining a trained object classification model in the following ways:

[0229] The object classification model is invoked to obtain the feature differences among the summative features of multiple objects corresponding to the k-th training object; the feature differences are used to maximize the differences among the summative features of multiple objects corresponding to the k-th training object.

[0230] The model parameters of the object classification model are updated based on feature differences, feature summarization bias, and type prediction bias to obtain a trained object classification model.

[0231] Optionally, any second sample data in the target sample domain is represented as the j-th second sample data, where j is a positive integer less than or equal to the total number of second sample data in the target sample domain; the number of object summary features corresponding to a training object is multiple;

[0232] Feature matching module 13 calls the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, and obtains the predicted object type for the second sample data in the target sample domain, including:

[0233] The object classification model is invoked to extract features from the j-th second sample data, generating sample features for the j-th second sample data;

[0234] Feature matching is performed between the sample features of the j-th second sample data and the object summary features corresponding to each training object;

[0235] The object with the smallest feature distance to the sample features of the j-th second sample data is identified as the object type of the training object to which the feature belongs, and thus determined as the predicted object type of the object to which the j-th second sample data belongs.

[0236] Optionally, the deviation generation module 14 generates a type prediction deviation based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain, including:

[0237] Generate type prediction cross-entropy for the j-th second sample data based on the predicted object type of the j-th second sample data and the object type label carried by the j-th second sample data;

[0238] The type prediction loss function is determined based on the type prediction cross-entropy for the j-th second sample data; the type prediction loss function is used to reflect the type prediction bias.

[0239] Optionally, any first sample data in the source sample domain is represented as the i-th first sample data, where i is a positive integer less than or equal to the total number of first sample data in the source sample domain; the number of object summary features corresponding to a training object is multiple;

[0240] Feature calculation module 12 calls the object classification model to perform feature calculation between the first sample data in the source sample domain and the object summary features corresponding to N training objects, and obtains the feature summary bias for the object summary features, including:

[0241] The object classification model is invoked to extract features from the i-th first sample data, generating sample features for the i-th first sample data;

[0242] The training object to which the i-th first sample data belongs is determined as the target training object;

[0243] The summary feature of an object with the largest feature distance from the sample feature of the i-th first sample data among the multiple summary features of objects corresponding to the target training object is determined as the summary feature of the in-class object;

[0244] Among the N types of training objects, the training objects other than the target training object are determined as reference training objects;

[0245] The summary feature of an out-of-class object is determined as the summary feature of an object with the smallest feature distance to the sample feature of the i-th first sample data among the summaries of multiple objects corresponding to the reference training object.

[0246] A summary bias loss function is generated based on the summarized features of objects within the class and the summarized features of objects outside the class; the summary bias loss function is used to reflect the feature summary bias.

[0247] Optionally, the feature calculation module 12 generates the summary bias loss function based on the summarized features of objects within the class and the summarized features of objects outside the class in the following ways:

[0248] The feature distance between the sample features of the i-th first sample data and the summary features of objects within the class is determined as the first feature distance;

[0249] The feature distance between the sample features of the i-th first sample data and the summary features of out-of-class objects is determined as the second feature distance;

[0250] A summary bias loss function is generated based on the difference between the first feature distance and the second feature distance.

[0251] Optionally, the trained object classification model includes the object summary features corresponding to each trained object.

[0252] The aforementioned device 1 is also used for:

[0253] Obtain prediction data; the prediction data and the second sample data in the target sample domain have object features of the same modality as the training object;

[0254] The trained object classification model is invoked to extract features from the prediction data, generating prediction features for the prediction data.

[0255] Feature matching is performed between the predicted features and the summaries of the features corresponding to each trained object;

[0256] The object with the smallest feature distance between the matched and predicted features is identified as the training object to which the feature belongs and is then determined as the predicted training object.

[0257] The object type of the training object is determined as the object type of the object to which the prediction data belongs.

[0258] According to one embodiment of this application, Figure 3 The steps involved in training the object classification model shown can be derived from... Figure 11 The training apparatus 1 shown is used to execute the object classification model. For example, Figure 3 Step S101 shown can be performed by Figure 11 The sample acquisition module 11 in the middle is used to execute, Figure 3 Step S102 shown can be performed by Figure 11 The feature calculation module 12 in the middle is used to execute it. Figure 3 Step S103 shown can be performed by Figure 11 The feature matching module 13 in the middle is used to perform this. Figure 3 Step S104 shown can be derived from Figure 11 The deviation generation module 14 in the middle is used to execute this. Figure 3 Step S105 shown can be performed by Figure 11 The training module 15 in the middle is used to execute it.

[0259] This application can obtain first sample data from a source sample domain and second sample data from a target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data in the source sample domain and the second sample data in the target sample domain each possess object features of different modalities of the N training objects; both the first sample data in the source sample domain and the second sample data in the target sample domain carry object type labels for their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain; an object classification model is invoked to perform classification between the summative object features corresponding to the first sample data in the source sample domain and the N training objects. Feature calculation yields the feature summary bias for the summarized features of the objects. The model parameters of the object classification model include the summarized features corresponding to N training objects. The object classification model is called to perform feature matching between the second sample data in the target sample domain and the summarized features corresponding to each training object, obtaining the predicted object type of the object to which the second sample data belongs in the target sample domain. A type prediction bias is generated based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain. The model parameters of the object classification model are updated based on the feature summary bias and the type prediction bias, resulting in a trained object classification model. Therefore, the method proposed in this application can update the summarized features corresponding to various training objects through the source sample domain, and then match the second sample data with the summarized features corresponding to various training objects. This achieves feature matching between the class template (such as the summarized features) and the second sample data, rather than direct matching between the first and second sample data. This method trains the object classification model, resulting in higher classification accuracy.

[0260] According to one embodiment of this application, Figure 11 The modules in the training device 1 of the object classification model shown can be individually or entirely merged into one or more units, or some of these units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division. In practical applications, the function of one module can be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the training device 1 of the object classification model may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0261] According to one embodiment of this application, a general-purpose computer device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 11 The diagram shows a training apparatus 1 for an object classification model, and a training method for implementing the object classification model according to embodiments of this application. The computer program described above can be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the same medium, and run therein.

[0262] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a computer device provided in this application. Figure 12 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 12 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0263] exist Figure 12 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0264] Obtain the first sample data from the source sample domain and the second sample data from the target sample domain; both the source and target sample domains are used to train N types of training objects; the first sample data from the source sample domain and the second sample data from the target sample domain each have object features of different modalities of the N training objects; both the first sample data from the source sample domain and the second sample data from the target sample domain carry object type labels of their respective training objects; the number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain;

[0265] The object classification model is invoked to perform feature measurement between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and the feature summary bias for the object summary features is obtained; the model parameters of the object classification model include the object summary features corresponding to the N training objects.

[0266] The object classification model is invoked to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, so as to obtain the predicted object type of the object to which the second sample data in the target sample domain belongs;

[0267] Type prediction bias is generated based on the predicted object type of the object to which the second sample data belongs in the target sample domain and the object type label carried by the second sample data in the target sample domain;

[0268] The model parameters of the object classification model are updated based on the feature summarization bias and type prediction bias to obtain a trained object classification model.

[0269] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N; the source sample domain contains M first sample data of the k-th training object, where M is a positive integer less than N;

[0270] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0271] Select P first sample data points from M first sample data points; P is a positive integer less than M.

[0272] Based on M first sample data and P first sample data, generate P object summary features for the k-th training object.

[0273] Optionally, the processor 1001 can also be used to call the device control application stored in the memory 1005 to achieve:

[0274] The object classification model is invoked to extract features from the M first sample data, generating sample features corresponding to the M first sample data respectively;

[0275] Generate a first initial sample set to which each of the P first sample data belongs; each first initial sample set contains one of the P first sample data.

[0276] The first sample data that is not included in the P first sample data among the M first sample data is determined as undetermined sample data;

[0277] Based on the feature distances between the sample features of multiple undetermined sample data and the sample features of P first sample data, the multiple undetermined sample data are classified into P first initial sample sets, resulting in P first classified sample sets.

[0278] Generate P summary features of objects based on the P first-classification sample sets.

[0279] Optionally, the processor 1001 can also be used to call the device control application stored in the memory 1005 to achieve:

[0280] The sample features of the first sample data contained in each first classification sample set are respectively processed by feature averaging to obtain the first average sample features corresponding to each first classification sample set;

[0281] Generate a second initial sample set corresponding to each of the P first average sample features. This second initial sample set is empty.

[0282] Based on the feature distances between the sample features of M first sample data and the features of P first average sample data, the M first sample data are classified into P second initial sample sets, resulting in P second classification sample sets.

[0283] Generate P object summary features based on P sets of second-classification samples.

[0284] Optionally, the processor 1001 can also be used to call the device control application stored in the memory 1005 to achieve:

[0285] The sample features of the first sample data contained in each second classification sample set are respectively processed by feature averaging to obtain the second average sample features corresponding to each second classification sample set;

[0286] The features of the P second average samples are determined as the summary features of the P objects.

[0287] Optionally, any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N, and the number of object summary features corresponding to a training object is multiple.

[0288] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0289] The object classification model is invoked to obtain the feature differences among the summative features of multiple objects corresponding to the k-th training object; the feature differences are used to maximize the differences among the summative features of multiple objects corresponding to the k-th training object.

[0290] The model parameters of the object classification model are updated based on feature differences, feature summarization bias, and type prediction bias to obtain a trained object classification model.

[0291] Optionally, any second sample data in the target sample domain is represented as the j-th second sample data, where j is a positive integer less than or equal to the total number of second sample data in the target sample domain; the number of object summary features corresponding to a training object is multiple;

[0292] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0293] The object classification model is invoked to extract features from the j-th second sample data, generating sample features for the j-th second sample data;

[0294] Feature matching is performed between the sample features of the j-th second sample data and the object summary features corresponding to each training object;

[0295] The object with the smallest feature distance to the sample features of the j-th second sample data is identified as the object type of the training object to which the feature belongs, and thus determined as the predicted object type of the object to which the j-th second sample data belongs.

[0296] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0297] Generate type prediction cross-entropy for the j-th second sample data based on the predicted object type of the j-th second sample data and the object type label carried by the j-th second sample data;

[0298] The type prediction loss function is determined based on the type prediction cross-entropy for the j-th second sample data; the type prediction loss function is used to reflect the type prediction bias.

[0299] Optionally, any first sample data in the source sample domain is represented as the i-th first sample data, where i is a positive integer less than or equal to the total number of first sample data in the source sample domain; the number of object summary features corresponding to a training object is multiple;

[0300] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0301] The object classification model is invoked to extract features from the i-th first sample data, generating sample features for the i-th first sample data;

[0302] The training object to which the i-th first sample data belongs is determined as the target training object;

[0303] The summary feature of an object with the largest feature distance from the sample feature of the i-th first sample data among the multiple summary features of objects corresponding to the target training object is determined as the summary feature of the in-class object;

[0304] Among the N types of training objects, the training objects other than the target training object are determined as reference training objects;

[0305] The summary feature of an out-of-class object is determined as the summary feature of an object with the smallest feature distance to the sample feature of the i-th first sample data among the summaries of multiple objects corresponding to the reference training object.

[0306] A summary bias loss function is generated based on the summarized features of objects within the class and the summarized features of objects outside the class; the summary bias loss function is used to reflect the feature summary bias.

[0307] Optionally, the processor 1001 can also be used to call the device control application stored in the memory 1005 to achieve:

[0308] The feature distance between the sample features of the i-th first sample data and the summary features of objects within the class is determined as the first feature distance;

[0309] The feature distance between the sample features of the i-th first sample data and the summary features of out-of-class objects is determined as the second feature distance;

[0310] A summary bias loss function is generated based on the difference between the first feature distance and the second feature distance.

[0311] Optionally, the trained object classification model includes the object summary features corresponding to each trained object.

[0312] Processor 1001 can also be used to call device control applications stored in memory 1005 to achieve:

[0313] Obtain prediction data; the prediction data and the second sample data in the target sample domain have object features of the same modality as the training object;

[0314] The trained object classification model is invoked to extract features from the prediction data, generating prediction features for the prediction data.

[0315] Feature matching is performed between the predicted features and the summaries of the features corresponding to each trained object;

[0316] The object with the smallest feature distance between the matched and predicted features is identified as the training object to which the feature belongs and is then determined as the predicted training object.

[0317] The object type of the training object is determined as the object type of the object to which the prediction data belongs.

[0318] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 The description of the training method for the object classification model in the corresponding embodiment can also be performed as described above. Figure 11 The description of the training device 1 for the object classification model in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0319] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the training device 1 for the object classification model mentioned above. The computer program includes program instructions, which, when executed by a processor, enable the execution of the aforementioned... Figure 3 The description of the training method for the object classification model in the corresponding embodiments is omitted here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0320] As an example, the above program instructions can be deployed and executed on a single computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed across multiple locations and interconnected via a communication network. Multiple computer devices distributed across multiple locations and interconnected via a communication network can form a blockchain network.

[0321] The aforementioned computer-readable storage medium can be a training device for the object classification model provided in any of the foregoing embodiments or an internal storage unit of the aforementioned computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0322] This application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 3 The training method for the object classification model described in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0323] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0324] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0325] The methods and related apparatuses provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0326] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A training method for an object classification model, characterized in that, The method includes: The process involves acquiring first sample data from a source sample domain and second sample data from a target sample domain. Both the source and target sample domains are used to train N types of training objects. The first sample data from the source domain and the second sample data from the target sample domain each possess object features of different modalities of the N training objects. Different modalities include different sample data acquisition methods or different data formats, including combinations of two or three of the following: text data format, image data format, and audio data format. Both the first sample data from the source domain and the second sample data from the target sample domain carry object type labels for their respective training objects. The number of second sample data points in the target sample domain is less than the number of first sample data points in the source sample domain. The object classification model is invoked to perform feature measurement between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, respectively, to obtain the feature summary bias for the object summary features; the model parameters of the object classification model include the object summary features corresponding to the N training objects respectively; the object summary feature corresponding to a training object is one or more class features extracted from that training object; The object classification model is invoked to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, thereby obtaining the predicted object type of the object to which the second sample data in the target sample domain belongs; A type prediction bias is generated based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain; The model parameters of the object classification model are updated based on the feature summary bias and the type prediction bias to obtain a trained object classification model. The trained object classification model is used to predict the object type of the object in the prediction data provided by the terminal device.

2. The method according to claim 1, characterized in that, Any one of the N training objects is represented as the k-th training object, where k is a positive integer less than or equal to N; the source sample domain contains M first sample data of the k-th training object, where M is a positive integer less than N; The process of obtaining the object summary features corresponding to each training object in the object classification model includes: Select P first sample data from the M first sample data; P is a positive integer less than M; Based on the M first sample data and the P first sample data, P object summary features of the k-th training object are generated.

3. The method according to claim 2, characterized in that, The step of generating P summary features for the k-th training object based on the P first sample data includes: The object classification model is invoked to extract features from the M first sample data, generating sample features corresponding to the M first sample data respectively; Generate a first initial sample set to which each of the P first sample data belongs; a first initial sample set contains one of the P first sample data. The first sample data, excluding the P first sample data, among the M first sample data are determined as undetermined sample data; Based on the feature distances between the sample features of multiple undetermined sample data and the sample features of the P first sample data, the multiple undetermined sample data are classified into P first initial sample sets to obtain P first classified sample sets. The P summaries of object features are generated based on the P sets of first-classification samples.

4. The method according to claim 3, characterized in that, The step of generating the P object summary features based on the P first classification sample sets includes: The sample features of the first sample data contained in each first classification sample set are respectively processed by feature averaging to obtain the first average sample features corresponding to each first classification sample set. Generate a second initial sample set corresponding to each of the P first average sample features. This second initial sample set is empty. Based on the feature distances between the sample features of the M first sample data and the P first average sample features, the M first sample data are classified into P second initial sample sets to obtain P second classification sample sets. The P object summary features are generated based on the P second classification sample sets.

5. The method according to claim 4, characterized in that, The step of generating the P object summary features based on the P second classification sample sets includes: The sample features of the first sample data contained in each second classification sample set are respectively processed by feature averaging to obtain the second average sample features corresponding to each second classification sample set. The features of the P second average samples are determined as the summary features of the P objects.

6. The method according to claim 1, characterized in that, Any one of the N training objects is represented as the kth training object, where k is a positive integer less than or equal to N, and the number of object summary features corresponding to a training object is multiple. The step of updating the model parameters of the object classification model based on the feature summarization bias and the type prediction bias to obtain the trained object classification model includes: Call the object classification model to obtain the feature differences between the summary features of multiple objects corresponding to the kth training object; The feature difference is used to maximize the difference between the summarized features of multiple objects corresponding to the k-th training object; The model parameters of the object classification model are updated based on the feature differences, the feature summarization bias, and the type prediction bias to obtain a trained object classification model.

7. The method according to claim 1, characterized in that, Any second sample data in the target sample domain is represented as the j-th second sample data, where j is a positive integer less than or equal to the total number of second sample data in the target sample domain; the number of object summary features corresponding to a training object is multiple; The step of calling the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, and obtaining the predicted object type for the second sample data in the target sample domain, includes: The object classification model is invoked to extract features from the j-th second sample data, generating sample features for the j-th second sample data; Feature matching is performed between the sample features of the j-th second sample data and the object summary features corresponding to each training object; The object with the smallest feature distance to the sample features of the j-th second sample data is identified as the object type of the training object to which the feature belongs, and is determined as the predicted object type of the object to which the j-th second sample data belongs.

8. The method according to claim 7, characterized in that, The step of generating type prediction bias based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain includes: Generate type prediction cross-entropy for the j-th second sample data based on the predicted object type of the j-th second sample data and the object type label carried by the j-th second sample data; The type prediction loss function is determined based on the type prediction cross-entropy for the j-th second sample data; the type prediction loss function is used to reflect the type prediction bias.

9. The method according to claim 1, characterized in that, Any first sample data in the source sample domain is represented as the i-th first sample data, where i is a positive integer less than or equal to the total number of first sample data in the source sample domain; The number of object summary features corresponding to a training object is multiple; The object classification model performs feature measurement between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, respectively, to obtain the feature summary bias for the object summary features, including: The object classification model is invoked to extract features from the i-th first sample data, generating sample features for the i-th first sample data; The training object to which the i-th first sample data belongs is determined as the target training object; The object summary feature with the largest feature distance to the sample feature of the i-th first sample data among the multiple object summary features corresponding to the target training object is determined as the in-class object summary feature. The training objects other than the target training object among the N types of training objects are determined as reference training objects; The object summary feature with the smallest feature distance to the sample feature of the i-th first sample data among the multiple object summary features corresponding to the reference training object is determined as the out-of-class object summary feature. The summary deviation loss function is generated based on the summary features of the in-class objects and the summary features of the out-of-class objects; the summary deviation loss function is used to reflect the feature summary deviation.

10. The method according to claim 9, characterized in that, The step of generating the summary bias loss function based on the summary features of the in-class objects and the summary features of the out-of-class objects includes: The feature distance between the sample features of the i-th first sample data and the summary features of the in-class objects is determined as the first feature distance; The feature distance between the sample features of the i-th first sample data and the summary features of the out-of-class objects is determined as the second feature distance; The summary bias loss function is generated based on the difference between the first feature distance and the second feature distance.

11. The method according to claim 1, characterized in that, The trained object classification model contains the object summary features corresponding to each of the trained objects; The method further includes: Acquire prediction data; the prediction data and the second sample data in the target sample domain have object features of the same modality as the training object; The trained object classification model is invoked to extract features from the prediction data, generating prediction features for the prediction data. Feature matching is performed between the predicted features and the object summary features corresponding to each trained object; The training object to which the summarizing feature belongs is determined as the prediction training object; The object type of the prediction training object is determined to be the object type of the object to which the prediction data belongs.

12. A training device for an object classification model, characterized in that, The device includes: The sample acquisition module is used to acquire first sample data from a source sample domain and second sample data from a target sample domain. Both the source and target sample domains are used to train N types of training objects. The first sample data from the source sample domain and the second sample data from the target sample domain each possess object features of different modalities of the N training objects. Different modalities include different sample data acquisition methods or different data formats, including combinations of two or three of the following: text data format, image data format, and audio data format. Both the first sample data from the source sample domain and the second sample data from the target sample domain carry object type labels for their respective training objects. The number of second sample data in the target sample domain is less than the number of first sample data in the source sample domain. The feature calculation module is used to call the object classification model to perform feature calculation between the first sample data in the source sample domain and the object summary features corresponding to the N training objects, and obtain the feature summary bias for the object summary features; the model parameters of the object classification model include the object summary features corresponding to the N training objects; the object summary feature corresponding to a training object is one or more types of features extracted from that training object. The feature matching module is used to call the object classification model to perform feature matching between the second sample data in the target sample domain and the object summary features corresponding to each training object, so as to obtain the predicted object type of the object to which the second sample data in the target sample domain belongs; The deviation generation module is used to generate a type prediction deviation based on the predicted object type of the object to which the second sample data in the target sample domain belongs and the object type label carried by the second sample data in the target sample domain. The training module is used to update the model parameters of the object classification model based on the feature summary bias and the type prediction bias, so as to obtain the trained object classification model. This module is used to predict the object type of the object in the prediction data provided by the terminal device using a trained object classification model.

13. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-11.

14. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-11.

Images