Neural network training method, data processing method and equipment

Abstract

The embodiment of the invention discloses a neural network training method, a data processing method and equipment, and the method can use an artificial intelligence technology for making a decision, and comprises the steps: generating first feature information and second feature information corresponding to a first training sample through a feature obtaining network according to the feature information of the first training sample, the first feature information comprises features of information associated with the target attribute in the first training sample; the feature acquisition network is trained, the first loss function item indicates the similarity between prediction category information and expected category information, and the expected category information indicates the correct category of information associated with the target attribute in the first training sample; the purpose of training by adopting the second loss function item comprises reducing the similarity between the first feature information and the second feature information. The features of the information associated with and not associated with the target attribute causing the deviation are extracted respectively, so that the accuracy of the finally obtained prediction decision information is improved.

Description

[0001] This application is a divisional application. The original application has the application number 202210613415.0 and the original application date is May 31, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence, and in particular to a method for training a neural network, a method for processing data, and an apparatus. Background Technology

[0003] With the development of the Artificial Intelligence (AI) era, neural networks are being used more and more widely to help people make decisions. For example, in news recommendation systems, neural networks can be used to determine whether to recommend specific news items; in educational recommendation systems, they can be used to determine whether to recommend specific courses; and they can be used to determine whether to grant loans to specific applicants, and so on.

[0004] However, because the training data we collect may contain biases, the predictive decision information output by the trained neural network during the inference phase will also be biased, and may even amplify these biases. For example, in an educational recommendation system, because users tend to search for courses based on the provider's name, the collected training data will contain more feedback data on courses offered by mainstream providers and less feedback data on courses offered by niche providers. The neural network trained on this data will consider the provider when determining recommended courses; that is, the neural network learns the biases present in the training data. During the inference phase, courses offered by mainstream providers will have a significantly higher probability of being recommended. However, the quality of courses offered by specific providers may not be good, meaning the predictive decision information output by the neural network during the inference phase will also be biased.

[0005] As can be seen from the above description, the predictive decision information output by the neural network currently used for decision-making is biased, and a solution that can improve the predictive decision information output by the neural network is urgently needed. Summary of the Invention

[0006] This application provides a method for training a neural network, a method for processing data, and an apparatus.

[0007] To address the aforementioned technical problems, the embodiments of this application provide the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a method for training a neural network, which can use artificial intelligence technology for decision-making. The method includes: a training device inputting a first training sample into a feature acquisition network; extracting features from the first training sample through the feature acquisition network to obtain feature information of the first training sample; and generating first feature information and second feature information corresponding to the first training sample through the feature acquisition network based on the feature information of the first training sample. The first feature information corresponding to the first training sample includes features of information in the first training sample that are associated with a target attribute; the second feature information corresponding to the first training sample includes features of information in the first training sample that are not associated with any target attribute, meaning the training target includes different information in the obtained first and second feature information. Furthermore, those skilled in the art can determine the target attribute based on factors in the current task that cause deviations in the predictive decision information. As an example, if a trained neural network is used to determine whether to recommend a course, and factors that could lead to bias in the prediction decision include the provider of the course, then at least one target attribute used in the training phase of the neural network could include the provider. If the first training sample is a course, the information associated with the target attribute in the first training sample could include a watermark indicating the provider of the course, information about the provider on the course cover, or other information. As another example, if a trained neural network is used to determine whether a face in an image has curly hair, and factors that could lead to bias in the prediction decision include the gender of the person in the image, then at least one target attribute used in the training phase of the neural network could include the user's gender. The information associated with the target attribute in the face image (i.e., the first training sample) could include image information of the neck portion of the face image, which could be used to determine whether the user has an Adam's apple, etc.

[0009] The training device performs a classification operation based on the first feature information corresponding to the first training sample to obtain predicted category information. The predicted category information indicates the predicted category of the first feature information corresponding to the first training sample, that is, the first predicted category information indicates the predicted category corresponding to the information associated with the target attribute in the first training sample. The predicted category is included in multiple categories corresponding to the target attribute. The feature acquisition network is trained according to the first loss function to obtain the trained feature acquisition network. For example, if the neural network is used to determine whether a course is recommended, and the target attribute is the supplier, then the multiple categories corresponding to the target attribute can include supplier A, supplier B, supplier C, and supplier D, etc. For another example, if the neural network is used to determine whether a person in an image has curly hair, and the target attribute is gender, then the multiple categories corresponding to the target attribute can include male and female.

[0010] The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information. Furthermore, the second loss function term can directly calculate the similarity between the first feature information and the second feature information, or it can also use the similarity between other information.

[0011] In this implementation, the feature information of the first training sample is decomposed by a feature acquisition network to obtain first feature information and second feature information corresponding to the first training sample. The first feature information corresponding to the first training sample includes features of information associated with the target attribute in the first training sample. A classification operation is performed based on the first feature information corresponding to the first training sample to obtain predicted category information. The predicted category information indicates the predicted category of the first feature information corresponding to the first training sample, and the predicted category is included in multiple categories corresponding to the target attribute. The feature acquisition network is trained according to a first loss function to obtain a trained feature acquisition network. The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of information associated with the target attribute in the first training sample. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information. Through the aforementioned scheme, the trained feature acquisition network can acquire features of information related to the target attribute in the input data, and features of information not related to the target attribute in the input data. Thus, users can determine the target attribute based on the reasons for the deviation in the prediction decision information in the current task, and extract the features of the information related to the target attribute that caused the deviation and the features of the information not related to the target attribute respectively. This helps to reduce the difficulty of the subsequent decision-making process and improve the accuracy of the final prediction decision information.

[0012] In one possible implementation of the first aspect, the method further includes: the training device combining the first feature information and the second feature information corresponding to the first training sample to obtain the first combined feature; the aforementioned "combination" operation can be any one or more of the following operations: splicing, addition or other types of operations; the first combined feature is input into the first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample.

[0013] The training device trains the feature acquisition network according to the first loss function, including: the training device trains the feature acquisition network and the first classification network according to the first loss function, wherein the first loss function further includes a third loss function term, the third loss function term indicating the similarity between the first predicted decision information and the expected decision information corresponding to the first training sample.

[0014] In this implementation, during the training of the feature acquisition network, the purpose of training the feature acquisition network is not only to accurately obtain the features associated with the target attribute from the feature information of the first training sample, but also to combine the first feature information and the second feature information to obtain the first combined feature, and a third loss function term is introduced. The purpose of using the third loss function term for training is to improve the accuracy of the prediction decision information obtained based on the first combined feature, which is conducive to further improving the accuracy of the prediction decision information obtained in the inference stage.

[0015] In one possible implementation of the first aspect, after the training device trains the feature acquisition network and the first classification network to obtain the trained feature acquisition network and the trained first classification network, the method further includes: the training device acquiring third feature information and combining the second feature information and the third feature information corresponding to the second training sample to obtain a second combined feature; wherein the third feature information and the first feature information corresponding to the second training sample have the same data size but different data content. Further, if the first feature information is specifically represented as an N-dimensional tensor, then the third feature information is also represented as an N-dimensional tensor, and the length of the third feature information and the first feature information is the same in each of the aforementioned N dimensions. As an example, if the first feature information is specifically represented as a vector, then the third feature information is also represented as a vector, and the length of the third feature information is the same as that of the first feature information; if the first feature information is specifically represented as a matrix, then the third feature information is also represented as a matrix, and the length and width of the third feature information are the same as those of the first feature information, etc. The difference between the data content of the third feature information and the first feature information corresponding to the second training sample means that the data content of the third feature information and the first feature information corresponding to the second training sample are not completely the same, that is, there are different data in the third feature information and the first feature information corresponding to the second training sample.

[0016] The training device inputs the second combined features into the trained first classification network to obtain the second prediction decision information output by the trained first classification network corresponding to the second training sample; it then inputs the second combined features into the second classification network to obtain the third prediction decision information output by the second classification network corresponding to the second training sample; the training device trains the second classification network according to the second loss function to obtain the trained second classification network, and the trained feature acquisition network and the trained second classification network belong to the same target neural network; both the "second prediction decision information" and the "third prediction decision information" are decision information, and the content indicated by the "second prediction decision information" and the "third prediction decision information" depends on the type of target task performed by the target neural network.

[0017] The second loss function includes a fourth loss function term and a fifth loss function term. The fourth loss function term indicates the similarity between the second predicted decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicates the similarity between the second predicted decision information and the third predicted decision information.

[0018] In this implementation, the third feature information and the first feature information have the same data size but different data content. The combined feature obtained by combining the first feature information and the second feature information corresponding to the second training sample is different from the second combined feature, which means that the features of the information associated with the target attribute in the second training sample have changed. However, the training objective also includes generating the original expected decision information of the second training sample. That is, training data that does not exist is added during the training stage, and the training objective includes that the expected decision information can be obtained based on the information of target attributes of different categories. Through the above scheme, not only is the diversity of training data increased, but it also helps to reduce the dependence of the trained neural network on the information associated with the target attribute in the input data, and pays more attention to the information related to the task being performed. This is conducive to improving the accuracy of the output prediction decision information and improving the fairness of the prediction decision information obtained for the groups pointed to by different categories of target attributes.

[0019] In one possible implementation of the first aspect, the training device acquires the third feature information by: generating first feature information corresponding to the second training sample through the trained feature acquisition network; and performing a weighted summation of the first feature information corresponding to the second training sample and the perturbation information to obtain the third feature information, wherein the weight value of the perturbation information is adjustable. Further, the perturbation information can be information randomly generated by the training device, or it can include the gradient corresponding to the first loss function term. The perturbation information can also be obtained through other methods, etc., which are not exhaustively listed here.

[0020] In this implementation, the feature information of the second training sample is first decomposed, and then the features of the information associated with the target attribute are intervened to obtain a training sample that is the opposite of the second training sample. If the weight value of the perturbation information is larger, the similarity between the third feature information and the first feature information is lower, which is more conducive to improving the fairness of the obtained prediction decision information. The accuracy and fairness of the obtained prediction decision information can be balanced by adjusting the weight value of the perturbation information.

[0021] In one possible implementation of the first aspect, the method further includes: a training device performing a classification operation based on second feature information corresponding to the first training sample to obtain fourth prediction decision information corresponding to the first training sample; wherein the first loss function further includes a sixth loss function term, the sixth loss function term indicating the similarity between the fourth prediction decision information and the expected decision information corresponding to the first training sample, and the second loss function term indicating the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term.

[0022] In this implementation, the gradient corresponding to the first loss function term is to enable the acquired first feature information to more accurately reflect the information related to the target attribute. The gradient corresponding to the sixth loss function term is to eliminate the interference of the target attribute related information included in the first training sample based on the acquired second feature information, thereby generating more accurate prediction decision information. The second loss function term adopts the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term, which can further improve the update efficiency of the weight parameters of the feature acquisition network and is conducive to improving the accuracy of the first and second feature information generated by the trained feature acquisition network.

[0023] In one possible implementation of the first aspect, the method is applied to any of the following scenarios: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in a target state, or determining whether to agree to the request of the applicant pointed to by the first training sample. Correspondingly, the various "decision information" in this scheme can be used to indicate any of the following information: whether to recommend the object pointed to by the first training sample, whether the object in the first training sample is in a target state, or whether to agree to the request of the applicant pointed to by the first training sample. This implementation provides multiple application scenarios for this method, improving the implementation flexibility of this scheme.

[0024] Secondly, embodiments of this application provide a data processing method that can use artificial intelligence technology for decision-making. The method includes: an execution device inputting data to be processed into a feature acquisition network; the feature acquisition network extracts features from the data to be processed to obtain feature information of the data to be processed; based on the feature information of the data to be processed, the feature acquisition network generates first feature information and second feature information, the first feature information including features of information in the data to be processed that are associated with a target attribute. The execution device combines the first feature information and the second feature information to obtain a first combined feature; the first combined feature is input into a classification network to obtain predictive decision information output by the classification network. The feature acquisition network is trained using a first loss function, which includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between predicted category information and expected category information. The predicted category information indicates the predicted category of information in the data input to the feature acquisition network that is associated with the target attribute, and the expected category information indicates the correct category of information in the data input to the feature acquisition network that is associated with the target attribute. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0025] In one possible implementation of the second aspect, the first loss function further includes a third loss function term, which indicates the similarity between the first predicted decision information and the expected decision information corresponding to the data of the input feature acquisition network, and the first predicted decision information indicates the decision corresponding to the data of the input feature acquisition network.

[0026] In the second aspect of this application, the execution device can also be used to execute the steps of the training device in the first aspect and various possible implementations of the first aspect. The specific implementation methods, the meanings of the terms, and the beneficial effects of the steps in the various possible implementations of the second aspect can all be found in the first aspect, and will not be repeated here.

[0027] Thirdly, embodiments of this application provide a neural network training device that can use artificial intelligence technology for decision-making. The neural network training device includes: a feature extraction module, used to input a first training sample into a feature acquisition network, and extract features from the first training sample through the feature acquisition network to obtain feature information of the first training sample; a generation module, used to generate first feature information and second feature information corresponding to the first training sample through the feature acquisition network based on the feature information of the first training sample, wherein the first feature information corresponding to the first training sample includes features of information related to a target attribute in the first training sample; and a first classification module, used to perform classification based on the first feature information corresponding to the first training sample. A classification operation is performed to obtain predicted category information, which indicates the predicted category of the first feature information corresponding to the first training sample. The predicted category is included in multiple categories corresponding to the target attribute. A training module is used to train the feature acquisition network according to a first loss function to obtain the trained feature acquisition network. The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0028] In the third aspect of this application, the training device for the neural network can also be used to execute the steps performed by the training device in the first aspect and various possible implementations of the first aspect. The specific implementation methods, the meanings of the terms, and the beneficial effects of the steps in the various possible implementations of the third aspect can all be found in the first aspect, and will not be repeated here.

[0029] Fourthly, embodiments of this application provide a data processing apparatus that can use artificial intelligence technology for decision-making. The data processing apparatus includes: a feature extraction module for inputting data to be processed into a feature acquisition network, and extracting features from the data to be processed through the feature acquisition network to obtain feature information of the data to be processed; a generation module for generating first feature information and second feature information through the feature acquisition network based on the feature information of the data to be processed, wherein the first feature information includes features of information in the data to be processed that are associated with a target attribute; a combination module for combining the first feature information and the second feature information to obtain a first combined feature; and a classification module for inputting the first combined feature into a classification network to obtain prediction decision information output by the classification network. The feature acquisition network is trained using a first loss function, which includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between predicted category information and expected category information. The predicted category information indicates the predicted category of information in the data input to the feature acquisition network that is associated with the target attribute, and the expected category information indicates the correct category of information in the data input to the feature acquisition network that is associated with the target attribute. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0030] In the fourth aspect of this application, the data processing apparatus can also be used to perform the steps of the execution device in the second aspect and various possible implementations of the second aspect. The specific implementation methods, the meanings of the terms, and the beneficial effects of the steps in the various possible implementations of the fourth aspect can be found in the first aspect, and will not be repeated here.

[0031] Fifthly, embodiments of this application provide a computer program product, which includes a program that, when run on a computer, causes the computer to perform the neural network training method described in the first aspect, or causes the computer to perform the data processing method described in the second aspect.

[0032] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the neural network training method described in the first aspect, or causes the computer to perform the data processing method described in the second aspect.

[0033] In a seventh aspect, embodiments of this application provide a training device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store a program, and the processor is used to execute the program in the memory, causing the training device to perform the neural network training method described in the first aspect above.

[0034] Eighthly, embodiments of this application provide an execution device, including a processor and a memory, the processor being coupled to the memory, the memory being used to store a program; the processor being used to execute the program in the memory, causing the execution device to perform the data processing method described in the second aspect above.

[0035] Ninthly, this application provides a chip system including a processor for supporting an execution device or communication device in implementing the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the execution device or communication device. This chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0036] Figure 1 A schematic diagram of the main framework of artificial intelligence provided in the embodiments of this application;

[0037] Figure 2a A system architecture diagram of the data processing system provided in this application embodiment;

[0038] Figure 2b A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0039] Figure 3 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0040] Figure 4 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0041] Figure 5 A comparative schematic diagram of the first feature information and the second feature information in the neural network training method provided in the embodiments of this application;

[0042] Figure 6 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0043] Figure 7 A schematic flowchart illustrating a neural network training method provided in an embodiment of this application;

[0044] Figure 8 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0045] Figure 9 A schematic diagram illustrating the beneficial effects of the neural network training method provided in the embodiments of this application;

[0046] Figure 10 A schematic diagram of a neural network training device provided in an embodiment of this application;

[0047] Figure 11 A schematic diagram of a data processing apparatus provided in an embodiment of this application;

[0048] Figure 12 A schematic diagram of the structure of the execution device provided in the embodiments of this application;

[0049] Figure 13 This is another schematic diagram of the structure of the training device provided in the embodiments of this application;

[0050] Figure 14 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0051] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0052] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0053] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1 , Figure 1 The diagram illustrates a structural framework for artificial intelligence (AI). The framework is further elaborated below along two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that AI brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed by technology) to the industrial ecosystem of the system.

[0054] (1) Infrastructure

[0055] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), neural-network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0056] (2) Data

[0057] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, as well as IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0058] (3) Data processing

[0059] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0060] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data by symbolizing and formalizing it.

[0061] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0062] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0063] (4) General ability

[0064] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0065] (5) Smart products and industry applications

[0066] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, autonomous driving, and smart cities.

[0067] The embodiments of this application can be applied to various application areas of artificial intelligence. Specifically, artificial intelligence technology can be used to solve decision-making problems in various application areas. As an example, in the field of smart cities, the predictive decision information output by the neural network can indicate whether to recommend specific news provided by a certain supplier; or, the predictive decision information output by the neural network can indicate whether to recommend specific courses provided by a certain supplier; or, the predictive decision information output by the neural network can indicate whether to recommend specific movies provided by a certain newspaper.

[0068] As another example, in the field of smart homes, the predictive decision information output by a neural network can indicate whether a person in an image has curly hair; or, whether a person in an image is smiling; or, whether a person in an image is attractive.

[0069] As another example, the predictive decision information output by a neural network can indicate whether to approve a loan request from a specific applicant, etc. The application scenarios of this application embodiment will not be exhaustively listed here.

[0070] To improve the accuracy of the predictive decision information output by the trained neural network, this application provides a method for training a neural network. Before describing the method, please refer to [the relevant documentation / reference needed]. Figure 2a , Figure 2a This application provides a system architecture diagram for a data processing system, in which... Figure 2a In the data processing system 200, there are training devices 210, database 220, execution devices 230, data storage systems 240 and client devices 250. The execution devices 230 include computing modules 231.

[0071] The database 220 stores a training dataset. The training device 210 generates a first model / rule 201 and iteratively trains the first model / rule 201 using the training dataset to obtain the trained first model / rule 201. The first model / rule 201 can be specifically represented as a neural network or as a non-neural network model. In this embodiment, the first model / rule 201 is described as a neural network. Furthermore, the first model / rule 201 may include a neural network for feature extraction from the input data.

[0072] For details, please refer to Figure 2b , Figure 2b This is a schematic flowchart of a neural network training method provided in an embodiment of this application. A1. The training device 210 inputs a first training sample into a feature acquisition network (i.e., an example of the first model / rule 201), and extracts features from the first training sample through the feature acquisition network to obtain feature information of the first training sample. A2. The training device 210 decomposes the feature information of the first training sample through the feature acquisition network to obtain first feature information and second feature information corresponding to the first training sample. The first feature information corresponding to the first training sample includes features of information in the first training sample associated with the target attribute. A3. The training device 210 performs a classification operation based on the first feature information corresponding to the first training sample to obtain predicted category information. The predicted category information indicates the predicted category of the first feature information corresponding to the first training sample, and the predicted category includes multiple categories corresponding to the target attribute. A4. The training device 210 trains the feature acquisition network according to the first loss function to obtain the trained feature acquisition network. The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0073] The trained first model / rule 201 obtained by training device 210 will be deployed to execution device 230. Execution device 230 can access data, code, etc. in data storage system 240, and can also store data, instructions, etc. in data storage system 240. Data storage system 240 can be located in execution device 230, or it can be an external storage device relative to execution device 230.

[0074] In some embodiments of this application, please refer to Figure 2aThe execution device 230 and the client device 250 can be separate independent devices. The execution device 230 is configured with an input / output (I / O) interface to interact with the client device 250. The "user" can input data to be processed through the client device 250. The client device 250 sends the data to be processed to the execution device 230 through the I / O interface. After the execution device 230 generates the prediction decision information corresponding to the data to be processed through the first machine learning model / rule 201 in the calculation module 231, it can return the aforementioned prediction decision information to the client device 250 through the I / O interface and provide it to the user.

[0075] It is worth noting that Figure 2a This is merely a schematic diagram of an architecture of the data processing system provided in this embodiment of the invention. The positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in some other embodiments of this application, the execution device 230 can be configured in the client device 250. As an example, when the client device is a mobile phone or tablet, the execution device 230 can be a module in the host CPU of the mobile phone or tablet used for array image processing. The execution device 230 can also be a graphics processing unit (GPU) or neural network processor (NPU) in the mobile phone or tablet. The GPU or NPU is mounted on the host processor as a coprocessor, and the host processor allocates tasks.

[0076] Based on the above description, the specific implementation process of the training and inference stages of the neural network provided in the embodiments of this application will now be described.

[0077] I. Training Phase

[0078] For specific details in the embodiments described in this application, please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a neural network training method provided in an embodiment of this application. The neural network training method provided in an embodiment of this application may include:

[0079] 301. The training device inputs the first training sample into the feature acquisition network, and extracts features from the first training sample through the feature acquisition network to obtain the feature information of the first training sample.

[0080] In this embodiment of the application, a training dataset is deployed on the training device. The training device can sample at least one first training sample from the training dataset, input the sampled first training sample into the feature acquisition network, and extract features from the first training sample through the feature acquisition network to obtain the feature information of the first training sample.

[0081] The feature acquisition network may include a first neural network module for feature extraction. The first neural network module may specifically adopt a convolutional neural network, a recurrent neural network, a residual neural network, or other types of neural networks. The specific selection can be based on the data type of the first training sample. For example, the first neural network module may adopt a residual neural network (ResNet)-18, ResNet-34, or other types of neural networks, etc., which will not be exhaustively listed here.

[0082] 302. The training device generates first feature information and second feature information corresponding to the first training sample through a feature acquisition network based on the feature information of the first training sample. The first feature information corresponding to the first training sample includes features of information related to the target attribute in the first training sample.

[0083] In this embodiment of the application, the training device can generate at least one first feature information and a second feature information corresponding to the first training sample through a feature acquisition network based on the feature information of the first training sample; wherein, each first feature information corresponding to the first training sample includes features of information in the first training sample that are associated with a target attribute, and the second feature information corresponding to the first training sample includes features of information in the first training sample that are not associated with each target attribute, that is, the training target includes different information in the obtained first feature information and second feature information.

[0084] Furthermore, technicians can determine the target attribute based on factors that cause bias in the predictive decision information during the current task. For example, if a trained neural network is used to determine whether to recommend a course, and factors causing bias in the predictive decision information include the course provider, then at least one target attribute used in the training phase of the neural network could include the provider. If the first training sample is a course, the information associated with the target attribute in the first training sample could include a watermark indicating the course provider, information about the provider on the course cover, or other information. As another example, if a trained neural network is used to determine whether a face in an image has curly hair, and factors causing bias in the predictive decision information include the gender of the person in the image, then at least one target attribute used in the training phase of the neural network could include the user's gender. The information associated with the target attribute in the face image (i.e., the first training sample) could include image information of the neck portion of the face image. This neck portion image information can be used to determine whether the user has an Adam's apple, etc. It should be understood that these examples are only for ease of understanding of the concepts of "target attribute" and "information associated with the target attribute in the training sample" and are not intended to limit this solution.

[0085] Specifically, in one implementation, the feature acquisition network may include at least one second neural network module and a third neural network module that correspond one-to-one with at least one target attribute. After generating feature information of the first training sample, the training device can obtain a first feature information corresponding to the first training sample from the feature information of the first training sample through each second neural network module, and obtain a second feature information corresponding to the first training sample from the feature information of the first training sample through the third neural network module.

[0086] In another implementation, the feature acquisition network may include a complete second neural network module. After generating the feature information of the first training sample, the training device can input the feature information of the first training sample into the second neural network module, and perform a decomposition operation through the second neural network module to obtain at least one first feature information and one second feature information corresponding to the first training sample output by the second neural network module.

[0087] It should be noted that feature acquisition networks can also take other structural forms. This description is only used to demonstrate the feasibility of this scheme and is not intended to limit this scheme.

[0088] 303. The training device performs a classification operation based on the first feature information corresponding to the first training sample to obtain first predicted category information. The first predicted category information indicates the predicted category of the first feature information corresponding to the first training sample. The predicted category is included in multiple categories corresponding to the target attribute.

[0089] In this embodiment of the application, the training device can input the first feature information corresponding to the first training sample into the first classifier, and perform a classification operation through the first classifier to obtain the first predicted category information generated by the first classifier; wherein, the first predicted category information indicates the predicted category of the first feature information corresponding to the first training sample, that is, the first predicted category information indicates the predicted category of the information associated with the target attribute in the first training sample, and the predicted category is included in the multiple categories corresponding to the target attribute.

[0090] As an example, if a neural network is used to determine whether a course is recommended, and the target attribute is the supplier, then the various categories corresponding to the target attribute can include supplier A, supplier B, supplier C, and supplier D, etc.; as another example, if a neural network is used to determine whether a person in an image has curly hair, and the target attribute is gender, then the various categories corresponding to the target attribute can include male and female; as yet another example, if a neural network is used to determine whether a movie is recommended, and the first training sample includes user ratings of the movie and multiple attribute information of the movie itself, and the target attribute can include the movie's producer, then the various categories corresponding to the target attribute can include producer A, producer B, producer C, and producer D, etc., and so on, without exhaustive listing.

[0091] 304. The training device performs a classification operation based on the second feature information corresponding to the first training sample to obtain the fourth prediction decision information corresponding to the first training sample.

[0092] In some embodiments of this application, the training device may also input the second feature information corresponding to the first training sample into the second classifier, and generate the fourth prediction decision information corresponding to the first training sample through the second classifier.

[0093] The trained feature acquisition network is used to perform the target task. The "fourth predictive decision information" is the predictive decision information output by the first classifier, and the content indicated by the "fourth predictive decision information" depends on the target task. Furthermore, the trained feature acquisition network can be applied to any of the following scenarios: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in the target state, or determining whether to agree to the request of the applicant pointed to by the first training sample.

[0094] Correspondingly, the "fourth predictive decision information" can be used to indicate any of the following: whether to recommend the object pointed to by the first training sample, whether the object in the first training sample is in the target state, whether to agree to the request of the applicant pointed to by the first training sample, or the "fourth predictive decision information" can also be used to indicate other types of information, etc.

[0095] Furthermore, as examples, the object pointed to by the first training sample can be news, courses, or other types of objects; the target state can be smiling, curly hair, attractiveness, or other types of states; and the applicant's request can be a loan request, a promotion request, or other types of requests. These examples are only for the purpose of understanding this scheme and are not intended to limit it. It should be noted that the content of the information indicated by the "fourth predictive decision information" is determined based on the content of the target task, and is not exhaustively listed here. This provides multiple application scenarios for this method, improving the implementation flexibility of this scheme.

[0096] 305. The training device combines the first feature information and the second feature information corresponding to the first training sample to obtain the first combined feature.

[0097] In this embodiment of the application, the training device can combine the first feature information and the second feature information corresponding to the first training sample to obtain the first combined feature. The aforementioned "combination" operation can be any one or more of the following operations: concatenation, addition, or other types of operations, etc., which are not exhaustively listed here.

[0098] Specifically, after obtaining the first feature information and the second feature information corresponding to the first training sample, the training device can directly perform a combination operation on the aforementioned first feature information and the second feature information; alternatively, it can preprocess the first feature information and / or the second feature information corresponding to the first training sample before performing the aforementioned combination operation. The aforementioned preprocessing operation may include normalization processing, processing through an activation function, multiplication with a preset weight value, or other processing methods, etc., which are not limited here.

[0099] 306. The training device inputs the first combined features into the first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample.

[0100] In this embodiment of the application, the training device can input the first combined features into the first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample; the meaning of "first prediction decision information" is similar to that of "fourth prediction decision information", the difference being that "first prediction decision information" is generated by the first classification network, while "fourth prediction decision information" is generated by the second classifier.

[0101] 307. The training device trains the feature acquisition network according to the first loss function, wherein the first loss function includes a first loss function term and a second loss function term, the first loss function term indicates the similarity between the first predicted category information and the first expected category information, the first expected category information indicates the correct category of the information associated with the target attribute in the first training sample, and the purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0102] In this embodiment, the training device can iteratively train the feature acquisition network and the first classifier according to the first loss function until a convergence condition is met, thus obtaining the trained feature acquisition network. The aforementioned convergence condition may include reaching a preset number of iterations, satisfying the convergence condition of the first loss function, or other convergence conditions, etc., which are not exhaustively listed here.

[0103] The first loss function may include a first loss function term and a second loss function term. The first loss function term indicates the similarity between first predicted category information and first expected category information. The first expected category information indicates the correct category of information associated with the target attribute in the first training sample. The purpose of training with the first loss function term includes increasing the similarity between the first predicted category information and the first expected category information. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0104] Furthermore, the first loss function term can be the cosine similarity, L1 similarity, L2 similarity, or other types of similarity between the first predicted category information and the first expected category information. Alternatively, the first loss function term can be obtained based on the Euclidean distance, cosine distance, Mahalanobis distance, or other types of distance between the first predicted category information and the first expected category information. The larger the distance between the first predicted category information and the first expected category information, the smaller the similarity between the first predicted category information and the first expected category information. It should be noted that the example of the first loss function term here is only for the convenience of understanding the first loss function term and is not intended to limit this scheme.

[0105] Specifically, steps 304 to 306 are optional. If steps 304 to 306 are not executed, the trained feature acquisition network can be obtained by repeatedly executing steps 301 to 303 and step 307 multiple times. The trained feature acquisition network can then be deployed to the execution device.

[0106] If steps 304 to 306 are performed, step 307 may include: the training device may iteratively train the feature acquisition network, the first classifier, the second classifier and the first classification network according to the first loss function until the convergence condition is met, so as to obtain the trained feature acquisition network and the trained first classification network.

[0107] More specifically, after generating the function value of the first loss function, the training device performs gradient differentiation on the function value of the first loss function and updates the weight parameters of the feature acquisition network, the first classifier, the second classifier, and the first classification network in reverse order to complete one training of the feature acquisition network and the first classification network.

[0108] The first loss function may further include a third loss function term and a sixth loss function term. The third loss function term indicates the similarity between the first predicted decision information and the expected decision information corresponding to the first training sample. The sixth loss function term indicates the similarity between the fourth predicted decision information and the expected decision information corresponding to the first training sample. The calculation methods for "the similarity between the first predicted decision information and the expected decision information corresponding to the first training sample" and "the similarity between the fourth predicted decision information and the expected decision information corresponding to the first training sample" can be referred to the calculation method for "the similarity between the first predicted category information and the first expected category information", which will not be elaborated here.

[0109] In this implementation, the second loss function term can directly calculate the similarity between the first feature information and the second feature information, or the second loss function term can also use the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term; the calculation method of the similarity between the aforementioned two gradients can be referred to the calculation method of "the similarity between the first predicted category information and the first expected category information", which will not be elaborated here.

[0110] To further understand this scheme, the calculation formula of the first loss function is shown below. Here, we take the cosine similarity between two gradients as an example to illustrate the similarity between the two gradients.

[0111] (1)

[0112] (2)

[0113] (3)

[0114] (4)

[0115] (5)

[0116] in, Let represent the first loss function, and n represent the number of training samples (i.e., a batch of training samples) that the training device obtains from the training dataset to train the feature acquisition network. Represents the second loss function term; This represents the second feature information (i.e., features in the training samples that are not related to the sensitive attributes). This represents the first feature information (i.e., the features of information related to sensitive attributes in the training samples). Represents the first classification network, This represents the expected decision information corresponding to the first training sample. This represents the third loss function term, which is also the first predicted decision information output by the first classification network. The similarity between them; Represents the second classifier. This represents the sixth loss function term, which is the similarity between the fourth predicted decision information and the expected decision information corresponding to the first training sample. Represents the first classifier. This represents the correct category (i.e., the first expected category information) of the information associated with the target attribute in the training samples. Represents the first loss function term, indicating the first predicted category information and The similarity between them; This represents the gradient corresponding to the sixth loss function term. This represents the gradient corresponding to the first loss function term. This represents the second loss function term. Here, we take the cosine similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term as an example. , and These represent three weight values. It should be understood that the example of the specific implementation of the first loss function here is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0117] If step 304 is executed, but steps 305 and 306 are not executed, then step 307 may include: the training device can iteratively train the feature acquisition network, the first classifier and the second classifier according to the first loss function until the convergence condition is met, and obtain the trained feature acquisition network.

[0118] The first loss function may include a first loss function term, a second loss function term, and a sixth loss function term. Furthermore, the second loss function term may directly calculate the similarity between the first feature information and the second feature information, or it may calculate the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term.

[0119] In this embodiment, the gradient corresponding to the first loss function term is to enable the acquired first feature information to more accurately reflect the information associated with the target attribute. The gradient corresponding to the sixth loss function term is to eliminate the interference of the target attribute associated information included in the first training sample based on the acquired second feature information, thereby generating more accurate prediction decision information. The second loss function term adopts the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term, which can further improve the update efficiency of the weight parameters of the feature acquisition network and is conducive to improving the accuracy of the first and second feature information generated by the trained feature acquisition network.

[0120] If step 304 is not executed, but steps 305 and 306 are executed, then step 307 may include: the training device can iteratively train the feature acquisition network, the first classifier, and the first classification network according to the first loss function until the convergence condition is met, so as to obtain the trained feature acquisition network and the trained first classification network.

[0121] The first loss function may include a first loss function term, a second loss function term, and a third loss function term; in this implementation, the second loss function term can calculate the similarity between the first feature information and the second feature information.

[0122] In this embodiment of the application, during the training of the feature acquisition network, the purpose of training the feature acquisition network not only includes accurately obtaining the features of the information associated with the target attribute from the feature information of the first training sample, but also combining the first feature information and the second feature information to obtain the first combined feature, and introducing a third loss function term. The purpose of using the third loss function term for training is to improve the accuracy of the prediction decision information obtained based on the first combined feature, which is conducive to further improving the accuracy of the prediction decision information obtained in the inference stage.

[0123] To understand this solution more intuitively, please refer to [link / reference]. Figure 4 and Figure 5 , Figure 4 This is a schematic flowchart of a neural network training method provided in an embodiment of this application. Figure 5 This is a comparative schematic diagram of the first feature information and the second feature information in the neural network training method provided in the embodiments of this application. (See also...) Figure 4 , Figure 4 Taking the predictive decision information output by a neural network to indicate whether a person in an image is smiling as an example, the training device can acquire the first training sample of a batch (patch). Figure 4 Taking four first training samples as an example, the expected decision information corresponding to each first training sample is obtained. Figure 4 (Taking the example of the first sample not smiling and the subsequent three smiling). The training device sequentially inputs the four first training samples into the feature extraction network to obtain the feature information of each first training sample. A decomposition operation is performed on the feature information of each first training sample to obtain the first feature information and the second feature information corresponding to each first training sample. The first feature information and the second feature information corresponding to each first training sample are concatenated to obtain the first combined feature corresponding to each first training sample. The training device inputs the first combined feature corresponding to each first training sample into the first classification network to obtain the first prediction decision information corresponding to each first training sample.

[0124] The training device can also perform classification operations based on the first feature information corresponding to each first training sample to obtain the first predicted category information corresponding to each first feature information; based on the first predicted category information, the first expected category information, the second feature information, the first predicted decision information, and the expected decision information, it generates the function value of the first loss function, calculates the gradient derivative of the first loss function, and updates the weight parameters of the feature extraction network and the first classification network in reverse, thereby completing multiple training iterations of the feature extraction network and the first classification network. It should be understood that... Figure 4 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0125] Please continue reading. Figure 5 , Figure 5 It is an image obtained after visualizing the first and second feature information. Figure 5 Taking the predictive decision information output by a neural network to indicate whether a person in an image has curly hair, with gender as the target attribute, as an example, Figure 5 As shown, the second feature information corresponding to the first training sample (i.e., features of information in the first training sample that are not related to the target attribute) carries more feature information of the hair region in the image, while the first feature information corresponding to the first training sample (i.e., features of information in the first training sample that are related to the target attribute) carries less feature information of the hair region and carries more features of information related to gender in the image. Comparing the second feature information and the first feature information corresponding to the first training sample, it is clear that they focus on different areas. Figure 5 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0126] 308. The training device generates second feature information corresponding to the second training sample through the feature acquisition network after training.

[0127] In some embodiments of this application, after the training device iteratively trains the feature acquisition network according to the first loss function, it can obtain the trained feature acquisition network. The training device can also obtain a second training sample from the training dataset and generate second feature information corresponding to the second training sample through the trained feature acquisition network (that is, the trained feature acquisition network obtained in steps 301 to 307). Optionally, the training device can generate first feature information and second feature information corresponding to the second training sample.

[0128] The specific implementation of the above steps can be found in the description of step 302. The concepts of "first feature information and second feature information corresponding to the second training sample" and "first feature information and second feature information corresponding to the first training sample" are similar, except that the first training sample and the second training sample are different training samples.

[0129] Optionally, after obtaining the second feature information corresponding to the second training sample, the training device can also perform a classification operation based on the second feature information corresponding to the second training sample to obtain the fifth prediction decision information corresponding to the second training sample. The concepts of "the fifth prediction decision information corresponding to the second training sample" and "the fourth prediction decision information corresponding to the first training sample" are similar. The specific method of obtaining the "fifth prediction decision information" can be referred to the description of the specific method of obtaining the "fourth prediction decision information" in the above steps, which will not be repeated here.

[0130] 309. The training device acquires third feature information. The data size of the third feature information is the same as that of the first feature information corresponding to the second training sample, but the data content is different.

[0131] In some embodiments of this application, the training device may also acquire third feature information; wherein the third feature information and the first feature information corresponding to the second training sample have the same data size but different data content.

[0132] Furthermore, if the first feature information is specifically represented as an N-dimensional tensor, then the third feature information is also represented as an N-dimensional tensor, and the length of the third feature information and the first feature information is the same in each of the aforementioned N dimensions. For example, if the first feature information is specifically represented as a vector, then the third feature information is also represented as a vector, and the length of the third feature information and the first feature information is the same; if the first feature information is specifically represented as a matrix, then the third feature information is also represented as a matrix, and the length and width of the third feature information and the first feature information are the same, and so on. This is not an exhaustive list.

[0133] The difference between the data content of the third feature information and the first feature information corresponding to the second training sample means that the data content of the third feature information and the first feature information corresponding to the second training sample are not completely the same, that is, there are different data in the third feature information and the first feature information corresponding to the second training sample.

[0134] Specifically, in one implementation, the training device can acquire the first feature information corresponding to the second training sample, and perform a weighted summation of the first feature information and the perturbation information corresponding to the second training sample to obtain the third feature information; wherein, the weight value of the perturbation information can be variable or fixed.

[0135] Furthermore, the perturbation information can be information randomly generated by the training device, or it can include the gradient corresponding to the first loss function term. The perturbation information can also be obtained through other means, etc., which will not be exhaustively listed here.

[0136] To provide a more intuitive understanding of this scheme, the following is an example of the formula for calculating the third feature information:

[0137] (6)

[0138] in, Represents the third characteristic information. Represents the first characteristic information. Represents disturbance information. The weights representing the perturbation information. From Obtained by uniform sampling in the middle, This represents the gradient corresponding to the first loss function term. Represents the first loss function term. This represents the predicted category information output by the first classifier, corresponding to the first feature information. This represents the expected category information corresponding to the first feature information (that is, the expected category information of the information associated with the target attribute in the second training sample). The L2 norm represents the gradient corresponding to the first loss function term; further, It can be constant or adjustable. It should be understood that the examples in equation (6) are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0139] In another implementation, the training device may also obtain the first feature information from the third training sample and determine the first feature information corresponding to the third training sample as the third feature information corresponding to the second training sample; the correct category of the information associated with the target attribute in the third training sample is different from the correct category of the information associated with the target attribute in the second training sample.

[0140] In another implementation, the training device can randomly acquire a third feature, and the first feature corresponding to the second training sample has the same data size as the randomly acquired third feature. It should be noted that the training device can also obtain the third feature in other ways, which will not be exhaustively listed here.

[0141] Optionally, after obtaining the first feature information corresponding to the second training sample, the training device can also perform a classification operation based on the first feature information corresponding to the second training sample to obtain second predicted category information. The second predicted category information indicates the predicted category of the first feature information corresponding to the second training sample. The concepts of "second predicted category information" and "first predicted category information" are similar. The specific method of obtaining "second predicted category information" can be referred to the description of the specific method of obtaining "first predicted category information" in the above steps, which will not be repeated here.

[0142] 310. The training device combines the second feature information and the third feature information corresponding to the second training sample to obtain the second combined feature.

[0143] 311. The training device inputs the second combined features into the trained first classification network to obtain the second prediction decision information output by the trained first classification network corresponding to the second training sample.

[0144] In this embodiment of the application, the specific implementation of steps 310 and 311 can be referred to the description in steps 305 and 306 above. The difference is that "second feature information corresponding to the first training sample" in steps 305 and 306 is replaced with "second feature information corresponding to the second training sample" in steps 310 and 311, and "first feature information corresponding to the first training sample" in steps 305 and 306 is replaced with "third feature information" in steps 310 and 311. The meaning of "second prediction decision information" is similar to the meaning of "fourth prediction decision information" above, and will not be repeated here.

[0145] 312. The training device inputs the second combined features into the second classification network to obtain the third prediction decision information output by the second classification network corresponding to the second training sample.

[0146] In some embodiments of this application, the training device can input the second combined features into the second classification network to obtain the third prediction decision information output by the second classification network corresponding to the second training sample. The meaning of the "third prediction decision information" is similar to that of the "fourth prediction decision information" mentioned above, and will not be repeated here.

[0147] 313. The training device trains the second classification network according to the second loss function, wherein the second loss function includes a fourth loss function term and a fifth loss function term, the fourth loss function term indicating the similarity between the second predicted decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicating the similarity between the second predicted decision information and the third predicted decision information.

[0148] In some embodiments of this application, the training device may also keep the weight parameters of the first classification network unchanged, and iteratively train the second classification network according to the second loss function until the convergence condition is met, thus obtaining the trained second classification network. The trained feature acquisition network and the trained second classification network belong to the same target neural network, which will be deployed on the execution device to perform the target task. The aforementioned target task may be any of the following tasks: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in the target state, determining whether to agree to the request of the applicant pointed to by the first training sample, or other types of tasks.

[0149] Optionally, the training device can also keep the weight parameters of the first classification network unchanged, and iteratively train the feature acquisition network and the second classification network according to the second loss function until the convergence condition is met, so as to obtain the trained second classification network and the retrained feature acquisition network. The trained second classification network and the retrained feature acquisition network belong to the above-mentioned target neural network.

[0150] The second loss function includes at least a fourth loss function term and a fifth loss function term. The fourth loss function term indicates the similarity between the second predicted decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicates the similarity between the second predicted decision information and the third predicted decision information.

[0151] Furthermore, the fourth loss function term can specifically employ cosine similarity, L1 similarity, L2 similarity, or other types of similarity between the second predicted decision information and the expected decision information corresponding to the second training sample. Alternatively, the fourth loss function term can be obtained based on Euclidean distance, cosine distance, Mahalanobis distance, or other types of distance between the second predicted decision information and the expected decision information corresponding to the second training sample. These are not exhaustive examples here. The specific form of the fifth loss function term can be found in the specific form of the fourth loss function term; it will not be elaborated upon here.

[0152] Optionally, the second loss function may also include a first loss function term and a second loss function term; the description of the "first loss function term and the second loss function term" can be found in step 307. It should be noted that the first loss function term and the second loss function term in step 307 are calculated based on the first training sample, while those in step 313 are calculated based on the second training sample.

[0153] Further optionally, the second loss function may also include a seventh loss function term, which indicates the similarity between the fifth predicted decision information and the expected decision information corresponding to the second training sample; then the second loss function term may adopt the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the seventh loss function term.

[0154] To further understand this scheme, the following is an example of the formula for calculating the second loss function:

[0155] (7)

[0156] (8)

[0157] (9)

[0158] (10)

[0159] in, represents the second loss function, and n represents the number of training samples (i.e., a batch of training samples) that the training device obtains from the training dataset to train the feature acquisition network and the second classification network. Represents the fourth loss function term. Represents the fifth loss function term. Represents the third characteristic information. This represents the third prediction decision information output by the second classification network, corresponding to the second training sample. This represents the expected decision information corresponding to the second training sample. This represents the second prediction decision information output by the first classification network after training, corresponding to the second training sample. The meaning of the other characters in equation (7) can be found in the above introduction to equation (1), and will not be repeated here.

[0160] To understand this solution more intuitively, please refer to [link / reference]. Figure 6 and Figure 7 , Figure 6 and Figure 7 Two schematic diagrams illustrating the training methods for neural networks provided in embodiments of this application. Figure 6 The main focus is on the training process of the target neural network, including the feature acquisition network and the second classification network, in the second training phase. Figure 6 Taking the predictive decision information output by a neural network to indicate whether a person in an image is smiling as an example, the training device can acquire a second training sample in a batch (patch). Figure 4 Taking four second training samples as an example, the expected decision information corresponding to each second training sample is obtained. Figure 4 (Taking the example of the first sample not smiling and the next three smiling). The training device inputs the four second training samples into the feature extraction network one after another to obtain the feature information of each second training sample. The feature information of each second training sample is decomposed to obtain the first feature information and the second feature information corresponding to each second training sample.

[0161] The training device combines the first feature information and perturbation information corresponding to each second training sample to obtain the third feature information corresponding to each second training sample. The third feature information and the second feature information corresponding to each second training sample are then concatenated to obtain the second combined feature corresponding to each second training sample.

[0162] The training device trains the first classification network and the second classification network respectively with the second combined features corresponding to each second training sample, and obtains the second prediction decision information output by the first classification network corresponding to the second training sample, and the third prediction decision information output by the second classification network corresponding to the second training sample.

[0163] The training device can generate a second loss function value based on the second prediction decision information, the third prediction decision information, and the expected decision information corresponding to each second training sample. The device then performs inverse differentiation on the second loss function value and updates the weight parameters of the feature extraction network and the first classification network to complete multiple training iterations of both networks. It should be noted that the specific meaning of the second loss function can be found in the above description. Figure 6 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0164] In this embodiment, the third feature information and the first feature information have the same data size but different data content. The combined feature obtained by combining the first feature information and the second feature information corresponding to the second training sample is different from the second combined feature, that is, the feature of the information associated with the target attribute in the second training sample has changed. However, the training objective also includes generating the original expected decision information of the second training sample. That is, training data that does not exist is added in the training stage, and the training objective includes that the expected decision information can be obtained based on the information of different categories of target attributes. Through the above scheme, not only is the diversity of training data increased, but it is also beneficial to reduce the dependence of the trained neural network on the information associated with the target attribute in the input data, and pay more attention to the information related to the task being performed. This is beneficial to improve the accuracy of the output prediction decision information and to improve the fairness of the prediction decision information obtained for the groups pointed to by different categories of target attributes.

[0165] In this embodiment, the feature information of the second training sample is first decomposed, and then the features of the information associated with the target attribute are intervened to obtain a training sample that is the opposite of the second training sample. If the weight value of the perturbation information is larger, the similarity between the third feature information and the first feature information is lower, which is more conducive to improving the fairness of the obtained prediction decision information. The accuracy and fairness of the obtained prediction decision information can be balanced by adjusting the weight value of the perturbation information.

[0166] To understand this solution more intuitively, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic flowchart illustrating a neural network training method provided in an embodiment of this application. Figure 7 As shown, the neural network training method provided in this application embodiment can be divided into a first training stage and a second training stage. In the first training stage, the training device inputs a first training sample into the feature extraction network, decomposes the extracted feature information to obtain the first feature information and the second feature information corresponding to the first training sample, and concatenates the first feature information and the second feature information corresponding to the first training sample to obtain the first combined feature. The training device can input the first combined feature into the first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample. The training device can generate the function value of the first loss function based on the information obtained in the aforementioned steps, and update the weight parameters of the feature acquisition network and the first classification network. The training device repeats the aforementioned steps to iteratively train the feature acquisition network and the first classification network to obtain the trained feature acquisition network and the trained first classification network.

[0167] In the second training phase, the training device inputs the second training sample into the feature extraction network and decomposes the extracted feature information to obtain the first feature information and the second feature information corresponding to the second training sample. The first feature information and the perturbation information corresponding to the second training sample are weighted and summed to obtain the third feature information. The second feature information and the third feature information corresponding to the second training sample are concatenated to obtain the second combined feature.

[0168] The training device inputs the combined features into the second classification network and the trained first classification network, respectively, to obtain the third prediction decision information corresponding to the second training sample output by the second classification network, and the second prediction decision information corresponding to the second training sample output by the trained first classification network. Based on the obtained information, the training device can generate the function value of the second loss function, and keep the weight parameters of the first classification network unchanged, while updating the weight parameters of the feature acquisition network and the first classification network. The training device repeats the above steps to iteratively train the feature acquisition network and the second classification network, resulting in a retrained feature acquisition network and a retrained second classification network. It should be understood that... Figure 7 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0169] In this embodiment, the trained feature acquisition network can acquire features of information related to the target attribute in the input data and features of information not related to the target attribute in the input data. Thus, the user can determine the target attribute based on the reasons for the deviation of the prediction decision information in the current task, and extract the features of the information related to the target attribute that caused the deviation and the features of the information not related to the target attribute respectively. This helps to reduce the difficulty of the subsequent decision-making process and improve the accuracy of the final prediction decision information.

[0170] II. Reasoning Stage

[0171] For specific details in the embodiments described in this application, please refer to [link / reference]. Figure 8 , Figure 8 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method provided in an embodiment of this application may include:

[0172] 801. The execution device inputs the data to be processed into the feature acquisition network, and the feature acquisition network extracts features from the data to be processed to obtain the feature information of the data to be processed.

[0173] 802. The execution device generates first feature information and second feature information through a feature acquisition network based on the feature information of the data to be processed. The first feature information includes features of information in the data to be processed that are associated with the target attribute.

[0174] In this embodiment, the specific implementation of steps 801 and 802 can be found in the above description. Figure 3 The descriptions in steps 301 and 302 of the corresponding embodiment are different in that "first training sample" in steps 301 and 302 is replaced with "data to be processed" in steps 801 and 802. The specific meanings of each term in steps 801 and 802 can be found above. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0175] 803. The execution device combines the first feature information and the second feature information to obtain the first combined feature.

[0176] 804. The execution device inputs the first combined features into the classification network to obtain the prediction decision information output by the classification network. The feature acquisition network is trained using a first loss function. The first loss function includes at least a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The predicted category information indicates the predicted category of the information associated with the target attribute in the data input to the feature acquisition network. The expected category information indicates the correct category of the information associated with the target attribute in the data input to the feature acquisition network. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0177] In this embodiment, the specific implementation of steps 803 and 804 can be found in the above description. Figure 3 The descriptions in steps 305 and 306 of the corresponding embodiments differ in that "first training sample" in steps 305 and 306 is replaced with "data to be processed" in steps 803 and 804. The specific meanings of the various terms in steps 803 and 804 can be found above. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0178] The feature acquisition network is trained using the first loss function. For details on the specific form of the feature acquisition network, please refer to [link / reference needed]. Figure 3 The classification network can specifically adopt the description in the corresponding embodiment. Figure 3 The first classification network in the corresponding embodiment can also be adopted. Figure 3 The second classification network in the corresponding embodiment can also be a classification network trained by other methods, etc., which will not be listed here.

[0179] Optionally, the feature extraction network and the classification network are trained using a first loss function and a second loss function. For the meaning of the first and second loss functions, please refer to [link / reference needed]. Figure 3 The descriptions in the corresponding embodiments will not be repeated here. For the specific meanings of each loss function term in "first loss function and second loss function", please refer to [the relevant documentation / reference needed]. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0180] In this embodiment, features of information associated with the target attribute in the data to be processed and features of information not associated with the target attribute in the data to be processed are obtained respectively. Since users can determine the target attribute based on the reasons that cause the deviation of the prediction decision information in the current task, the features of the information associated with the target attribute that causes the deviation and the features of the information not associated with the target attribute are extracted respectively. This is beneficial to the difficulty of the classification network in generating prediction decision information and to improve the accuracy of the final prediction decision information.

[0181] The following section will demonstrate the beneficial effects of the embodiments of this application, using experimental data. Figure 9 This is a schematic diagram illustrating the beneficial effects of the neural network training method provided in the embodiments of this application. Figure 9 Two sub-diagrams, one on the left and one on the right, are shown. Figure 9 The system uses movie producers as the target attribute, and categorizes them into mainstream and niche producers based on the number of movies they produce. The trained neural network is then used to determine whether to recommend a particular movie. Figure 9 The left and right sub-schematic diagrams each use a single broken line to illustrate the accuracy and fairness of the predicted decision information generated by the trained neural network obtained through the embodiments of this application. Three broken lines and a triangle are used to illustrate the accuracy and fairness of the predicted decision information generated by the trained neural network obtained through the control group method. A higher score for the accuracy index indicates higher accuracy of the generated predicted decision information; a lower score for the fairness index indicates better fairness of the generated predicted decision information in terms of the target attribute.

[0182] In some embodiments of this application, there is a second training stage, in which third feature information is obtained and the second feature information corresponding to the second training sample is combined with the third feature information to construct a training sample that does not actually exist. Figure 9 The rightmost point in the left and right sub-diagrams represents the accuracy and fairness of the predicted decision information without a second training phase. As shown in the figure, without a second training phase, regardless of... Figure 9 The left-hand diagram, or Figure 9 The right-hand diagram illustrates that the prediction decision information obtained based on the method of this application embodiment has the highest accuracy. Even with a second training phase, the prediction decision information obtained based on the method of this application embodiment still has the highest accuracy and demonstrates good fairness.

[0183] exist Figures 1 to 9 Based on the corresponding embodiments, in order to better implement the above-described solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 10 , Figure 10 This is a schematic diagram of a neural network training device provided in an embodiment of this application. The neural network training device 1000 includes: a feature extraction module 1001, used to input a first training sample into a feature acquisition network, and extract features from the first training sample through the feature acquisition network to obtain feature information of the first training sample; a generation module 1002, used to generate first feature information and second feature information corresponding to the first training sample through the feature acquisition network based on the feature information of the first training sample, wherein the first feature information corresponding to the first training sample includes features of information related to the target attribute in the first training sample; and a first classification module 1003, used to classify the first feature information corresponding to the first training sample. A classification operation is performed to obtain predicted category information, which indicates the predicted category of the first feature information corresponding to the first training sample. The predicted category is included in multiple categories corresponding to the target attribute. The training module 1004 is used to train the feature acquisition network according to the first loss function to obtain the trained feature acquisition network. The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0184] In one possible design, the neural network training device 1000 further includes: a combination module for combining the first feature information and the second feature information corresponding to the first training sample to obtain a first combined feature; a second classification module for inputting the first combined feature into a first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample; and a training module 1004 specifically used to train the feature acquisition network and the first classification network according to a first loss function, wherein the first loss function further includes a third loss function term, the third loss function term indicating the similarity between the first prediction decision information and the expected decision information corresponding to the first training sample.

[0185] In one possible design, the neural network training device 1000 further includes: an acquisition module for acquiring third feature information, wherein the third feature information has the same data size but different data content as the first feature information corresponding to the second training sample; a combination module for combining the second feature information and the third feature information corresponding to the second training sample to obtain a second combined feature; a second classification module for inputting the second combined feature into a trained first classification network to obtain second prediction decision information corresponding to the second training sample output by the trained first classification network; a second classification module for inputting the second combined feature into a second classification network to obtain third prediction decision information corresponding to the second training sample output by the second classification network; and a training module 1004 specifically for training the second classification network according to a second loss function to obtain a trained second classification network, wherein the trained feature acquisition network and the trained second classification network belong to the same target neural network; wherein the second loss function includes a fourth loss function term and a fifth loss function term, the fourth loss function term indicating the similarity between the second prediction decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicating the similarity between the second prediction decision information and the third prediction decision information.

[0186] In one possible design, the acquisition module is specifically used to: generate first feature information corresponding to the second training sample through the trained feature acquisition network; and perform a weighted summation of the first feature information and the perturbation information corresponding to the second training sample to obtain third feature information, wherein the weight value of the perturbation information is adjustable.

[0187] In one possible design, the second classification module is further configured to perform a classification operation based on the second feature information corresponding to the first training sample to obtain the fourth prediction decision information corresponding to the first training sample. The first loss function further includes a sixth loss function term, which indicates the similarity between the fourth prediction decision information and the expected decision information corresponding to the first training sample. The second loss function term indicates the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term.

[0188] In one possible design, the neural network training device 1000 is applied to any of the following scenarios: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in the target state, or determining whether to agree to the request of the applicant pointed to by the first training sample.

[0189] It should be noted that the information interaction and execution process between the modules / units in the neural network training device 1000 are different from those in this application. Figures 2b to 7The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0190] This application also provides a data processing apparatus; please refer to [link / reference]. Figure 11 , Figure 11 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 1100 includes: a feature extraction module 1101, used to input the data to be processed into a feature acquisition network, and extract features from the data to be processed through the feature acquisition network to obtain feature information of the data to be processed; a generation module 1102, used to generate first feature information and second feature information through the feature acquisition network based on the feature information of the data to be processed, wherein the first feature information includes features of information in the data to be processed that are associated with a target attribute; a combination module 1103, used to combine the first feature information and the second feature information to obtain a first combined feature; and a classification module 1. 104 is used to input the first combined features into a classification network to obtain the prediction decision information output by the classification network; wherein, the feature acquisition network is trained using a first loss function, which includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The predicted category information indicates the predicted category of the information associated with the target attribute in the data input to the feature acquisition network, and the expected category information indicates the correct category of the information associated with the target attribute in the data input to the feature acquisition network. The purpose of training with the second loss function term includes reducing the similarity between the first feature information and the second feature information.

[0191] In one possible design, the first loss function further includes a third loss function term, which indicates the similarity between the first predicted decision information and the expected decision information corresponding to the data of the input feature acquisition network, and the first predicted decision information indicates the decision corresponding to the data of the input feature acquisition network.

[0192] It should be noted that the information interaction and execution process between the modules / units in the data processing device 1100 are different from those in this application. Figure 8 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0193] The following describes an execution device provided in an embodiment of this application. Please refer to [link / reference]. Figure 12 , Figure 12This is a schematic diagram of an execution device provided in an embodiment of this application. Specifically, the execution device 1200 includes: a receiver 1201, a transmitter 1202, a processor 1203, and a memory 1204 (wherein the execution device 1200 may have one or more processors 1203). Figure 12 (Taking a processor as an example), processor 1203 may include application processor 12031 and communication processor 12032. In some embodiments of this application, receiver 1201, transmitter 1202, processor 1203 and memory 1204 may be connected via bus or other means.

[0194] Memory 1204 may include read-only memory and random access memory, and provides instructions and data to processor 1203. A portion of memory 1204 may also include non-volatile random access memory (NVRAM). Memory 1204 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0195] Processor 1203 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses in the diagram are referred to as the bus system.

[0196] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1203. The processor 1203 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1203 or by instructions in software form. The processor 1203 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1203 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1204. Processor 1203 reads the information in memory 1204 and, in conjunction with its hardware, completes the steps of the above method.

[0197] Receiver 1201 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1202 can be used to output digital or character information through the first interface; transmitter 1202 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1202 may also include a display device such as a display screen.

[0198] In this embodiment of the application, the application processor 12031 in the processor 1203 is used to execute... Figure 8 The corresponding embodiment describes a data processing method executed by the execution device. It should be noted that the specific methods of each step in the data processing method executed by the application processor 12031 differ from those described in this application. Figure 8 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 8 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0199] This application also provides a training device; please refer to [link / reference]. Figure 13 , Figure 13 This is a schematic diagram of a training device provided in an embodiment of this application. Specifically, the training device 1300 is implemented by one or more servers. The training device 1300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1322 (e.g., one or more processors) and memory 1332, and one or more storage media 1330 (e.g., one or more mass storage devices) for storing application programs 1342 or data 1344. The memory 1332 and storage media 1330 can be temporary or persistent storage. The program stored in the storage media 1330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the training device. Furthermore, the CPU 1322 may be configured to communicate with the storage media 1330 and execute the series of instruction operations in the storage media 1330 on the training device 1300.

[0200] The training device 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0201] In this embodiment, the central processing unit 1322 is used to execute... Figures 2b to 7 The training method for the neural network executed by the training device in the corresponding embodiment. It should be noted that the specific manner in which the central processing unit 1322 executes each step of the neural network training method differs from that in this application. Figures 2b to 7 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 2b to 7 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0202] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figure 8 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figures 2b to 7 The steps performed by the training device in the method described in the illustrated embodiment.

[0203] This application also provides a computer-readable storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform the aforementioned actions. Figure 8 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figures 2b to 7 The steps performed by the training device in the method described in the illustrated embodiment.

[0204] The neural network training device, data processing device, execution device, or training device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuitry. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to perform the aforementioned operations. Figures 2b to 7 The neural network training method described in the illustrated embodiment, or, to cause a chip within a training device to perform the above-described... Figure 8 The embodiment described herein illustrates a method for training a neural network. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0205] For details, please refer to Figure 14 , Figure 14 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 140. The NPU 140 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 140, which is controlled by the controller 1404 to extract matrix data from the memory and perform multiplication operations.

[0206] In some implementations, the arithmetic circuit 1403 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1403 is a two-dimensional pulsating array. The arithmetic circuit 1403 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.

[0207] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1402 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1401 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1408.

[0208] Unified memory 1406 is used to store input and output data. Weight data is directly transferred to weight memory 1402 via Direct Memory Access Controller (DMAC) 1405. Input data is also transferred to unified memory 1406 via DMAC.

[0209] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1409.

[0210] The Bus Interface Unit (BIU) 1410 is used by the instruction fetch memory 1409 to fetch instructions from external memory, and also by the memory access controller 1405 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0211] The DMAC is mainly used to move input data from external memory DDR to unified memory 1406, or to weight data to weight memory 1402, or to input data to input memory 1401.

[0212] The vector computation unit 1407 includes multiple arithmetic processing units that further process the output of the computation circuit as needed, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0213] In some implementations, the vector computation unit 1407 can store the processed output vector in the unified memory 1406. For example, the vector computation unit 1407 can apply linear and / or nonlinear functions to the output of the computation circuit 1403, such as performing linear interpolation on feature planes extracted by convolutional layers, or accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1407 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as activation input to the computation circuit 1403, for example, for use in subsequent layers of the neural network.

[0214] The instruction fetch buffer 1409 connected to the controller 1404 is used to store the instructions used by the controller 1404;

[0215] Unified memory 1406, input memory 1401, weighted memory 1402, and instruction fetch memory 1409 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0216] In the above embodiments, the operations of each layer in the neural network can be performed by the operation circuit 1403 or the vector calculation unit 1407.

[0217] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0218] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0219] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0220] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0221] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. A data processing method, characterized in that, The method includes: The first training sample is input into the feature acquisition network, and the feature acquisition network extracts features from the first training sample to generate first feature information and second feature information corresponding to the first training sample. The first feature information corresponding to the first training sample includes features of information related to the target attribute in the first training sample. The first training sample input to the feature acquisition network includes at least one of graphic data, image data, voice data, or text data. The first feature information and the second feature information corresponding to the first training sample are combined to obtain the first combined feature; The combined features are input into a first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample.

2. The method according to claim 1, characterized in that, The method further includes: A classification operation is performed based on the first feature information corresponding to the first training sample to obtain predicted category information. The predicted category information indicates the predicted category of the first feature information corresponding to the first training sample. The predicted category is included in multiple categories corresponding to the target attribute. The feature acquisition network is trained according to the first loss function to obtain the trained feature acquisition network; The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The second loss function term is used to reduce the similarity between the first feature information and the second feature information.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The feature acquisition network and the first classification network are trained according to the first loss function, wherein the first loss function includes a third loss function term, which indicates the similarity between the first predicted decision information and the expected decision information corresponding to the first training sample.

4. The method according to claim 3, characterized in that, After training the feature acquisition network and the first classification network to obtain the trained feature acquisition network and the trained first classification network, the method further includes: Obtain the third feature information, and combine the second feature information corresponding to the second training sample with the third feature information to obtain the second combined feature, wherein the third feature information and the first feature information corresponding to the second training sample have the same data size but different data content; The second combined features are input into the trained first classification network to obtain the second prediction decision information output by the trained first classification network corresponding to the second training sample; The combined features are input into the second classification network to obtain the third prediction decision information output by the second classification network corresponding to the second training sample. The second classification network is trained according to the second loss function to obtain the trained second classification network. The trained feature acquisition network and the trained second classification network belong to the same target neural network. The second loss function includes a fourth loss function term and a fifth loss function term. The fourth loss function term indicates the similarity between the second predicted decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicates the similarity between the second predicted decision information and the third predicted decision information.

5. The method according to claim 4, characterized in that, The acquisition of the third feature information includes: The trained feature acquisition network generates the first feature information corresponding to the second training sample; The first feature information and the perturbation information corresponding to the second training sample are weighted and summed to obtain the third feature information, wherein the weight value of the perturbation information is adjustable.

6. The method according to claim 2, characterized in that, The method further includes: A classification operation is performed based on the second feature information corresponding to the first training sample to obtain the fourth prediction decision information corresponding to the first training sample. The first loss function further includes a sixth loss function term, which indicates the similarity between the fourth prediction decision information and the expected decision information corresponding to the first training sample. The second loss function term indicates the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term.

7. The method according to claim 1 or 2, characterized in that, The method is applied to any of the following scenarios: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in a target state, or determining whether to agree to the request of the applicant pointed to by the first training sample.

8. A data processing method, characterized in that, The method includes: The data to be processed is input into a feature acquisition network, and the feature acquisition network extracts features from the data to be processed to generate first feature information and second feature information. The first feature information includes features of information related to the target attribute in the data to be processed. The data to be processed input into the feature acquisition network includes at least one of graphic data, image data, voice data, or text data. The first feature information and the second feature information are combined to obtain the first combined feature; The combined features are input into a classification network to obtain the prediction decision information output by the classification network.

9. The data processing method as described in claim 8, characterized in that, The feature acquisition network is trained using a first loss function, which includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between predicted category information and expected category information. The predicted category information indicates the predicted category of information associated with the target attribute in the data to be processed, and the expected category information indicates the correct category of information associated with the target attribute in the data to be processed. The second loss function term is used to reduce the similarity between the first feature information and the second feature information.

10. The method according to claim 9, characterized in that, The first loss function further includes a third loss function term, which indicates the similarity between the first predicted decision information and the expected decision information corresponding to the data to be processed, wherein the first predicted decision information indicates the decision corresponding to the data to be processed.

11. A data processing apparatus, characterized in that, The device includes: The feature extraction module is used to input a first training sample into a feature acquisition network, extract features from the first training sample through the feature acquisition network, and generate first feature information and second feature information corresponding to the first training sample. The first feature information corresponding to the first training sample includes features of information related to the target attribute in the first training sample. The first training sample input to the feature acquisition network includes at least one of graphic data, image data, voice data, or text data. The combination module is used to combine the first feature information and the second feature information corresponding to the first training sample to obtain the first combined feature; The second classification module is used to input the first combined features into the first classification network to obtain the first prediction decision information output by the first classification network corresponding to the first training sample.

12. The apparatus according to claim 11, characterized in that, The device further includes: The first classification module is used to perform a classification operation based on the first feature information corresponding to the first training sample to obtain predicted category information. The predicted category information indicates the predicted category of the first feature information corresponding to the first training sample. The predicted category is included in multiple categories corresponding to the target attribute. The training module is used to train the feature acquisition network according to the first loss function to obtain the trained feature acquisition network. The first loss function includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between the predicted category information and the expected category information. The expected category information indicates the correct category of the information associated with the target attribute in the first training sample. The second loss function term is used to reduce the similarity between the first feature information and the second feature information.

13. The apparatus according to claim 12, characterized in that, The device further includes: The training module is used to train the feature acquisition network and the first classification network according to the first loss function, wherein the first loss function includes a third loss function term, the third loss function term indicating the similarity between the first predicted decision information and the expected decision information corresponding to the first training sample.

14. The apparatus according to claim 13, characterized in that, The device further includes: The acquisition module is used to acquire third feature information, wherein the third feature information has the same data size as the first feature information corresponding to the second training sample but different data content; The combination module is further configured to combine the second feature information and the third feature information corresponding to the second training sample to obtain the second combined feature; The second classification module is further configured to input the second combined features into the trained first classification network to obtain the second prediction decision information output by the trained first classification network corresponding to the second training sample; The second classification module is further configured to input the second combined features into the second classification network to obtain the third prediction decision information output by the second classification network corresponding to the second training sample; The training module is specifically used to train the second classification network according to the second loss function to obtain the trained second classification network. The trained feature acquisition network and the trained second classification network belong to the same target neural network. The second loss function includes a fourth loss function term and a fifth loss function term. The fourth loss function term indicates the similarity between the second predicted decision information and the expected decision information corresponding to the second training sample, and the fifth loss function indicates the similarity between the second predicted decision information and the third predicted decision information.

15. The apparatus according to claim 14, characterized in that, The acquisition module is specifically used for: The trained feature acquisition network generates the first feature information corresponding to the second training sample; The first feature information and the perturbation information corresponding to the second training sample are weighted and summed to obtain the third feature information, wherein the weight value of the perturbation information is adjustable.

16. The apparatus according to claim 12, characterized in that, The second classification module is further configured to perform a classification operation based on the second feature information corresponding to the first training sample to obtain the fourth prediction decision information corresponding to the first training sample. The first loss function further includes a sixth loss function term, which indicates the similarity between the fourth prediction decision information and the expected decision information corresponding to the first training sample. The second loss function term indicates the similarity between the gradient corresponding to the first loss function term and the gradient corresponding to the sixth loss function term.

17. The apparatus according to claim 12, characterized in that, The device is applied to any of the following scenarios: determining whether to recommend the object pointed to by the first training sample, determining whether the object in the first training sample is in a target state, or determining whether to agree to the request of the applicant pointed to by the first training sample.

18. A data processing apparatus, characterized in that, The device includes: The feature extraction module is used to input the data to be processed into the feature acquisition network, and to extract features from the data to be processed through the feature acquisition network to generate first feature information and second feature information. The first feature information includes features of information related to the target attribute in the data to be processed. The data to be processed input into the feature acquisition network includes at least one of graphic data, image data, voice data, or text data. The combination module is used to combine the first feature information and the second feature information to obtain the first combined feature; The classification module is used to input the first combined features into the classification network to obtain the prediction decision information output by the classification network.

19. The data processing apparatus as claimed in claim 18, characterized in that, The feature acquisition network is trained using a first loss function, which includes a first loss function term and a second loss function term. The first loss function term indicates the similarity between predicted category information and expected category information. The predicted category information indicates the predicted category of information associated with the target attribute in the data to be processed, and the expected category information indicates the correct category of information associated with the target attribute in the data to be processed. The second loss function term is used to reduce the similarity between the first feature information and the second feature information.

20. The apparatus according to claim 19, characterized in that, The first loss function further includes a third loss function term, which indicates the similarity between the first predicted decision information and the expected decision information corresponding to the data input to the feature acquisition network, wherein the first predicted decision information indicates the decision corresponding to the data input to the feature acquisition network.

21. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as claimed in any one of claims 1 to 7, or causes the computer to perform the method as claimed in any one of claims 8 to 10.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as claimed in any one of claims 1 to 7, or causes the computer to perform the method as claimed in any one of claims 8 to 10.

23. A training device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the training device to perform the method as described in any one of claims 1 to 7.

24. An execution device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the execution device to perform the method as described in any one of claims 8 to 10.

Images