Neural network

Abstract

Computer-implemented method of generating a modified neural network and a trigger input comprises: obtaining a trained neural network; determining an identification neuron, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network; and generating, using the modified neural network, the trigger input such that: when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; wherein the identification neuron is such that, when the identification neuron is triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network.

Description

[0001]NEURAL NETWORK Technical Field The present invention concerns neural networks. More particularly, but not exclusively, this invention concerns computer-implemented methods for generating a modified neural network and a trigger input. Background Systems based on neural networks are increasingly common. Such systems includes devices that are deployed to locations outside of the control of the creator of the neural network. In such cases, there is a risk that the neural network can be copied by a third party and subsequently used in unauthorised ways or even resold as a product of the third party. Identifying that a neural network has been misappropriated in such a way is challenging, as it typically requires a direct comparison of the architecture, weights, and biases of the neural network. Furthermore, such misappropriation can be easily obscured by performing even small amounts of retraining of the neural network (for example, such that the values of the weights and biases in the neural network are changed to a sufficiently small extent that does not significantly impact the performance of the neural network). There is therefore a need for techniques to protect deployed neural networks against theft and to enable neural networks to be identified, such that any theft or copying of a neural network that does occur can be more readily detected and demonstrated. The present invention seeks to mitigate the above-mentioned problems. Alternatively or additionally, the present invention seeks to provide an improved computer-implemented method of generating a modified neural network and a trigger input. Summary According to a first aspect of the present invention, there is provided acomputer-implemented method of generating a modified neural network and a trigger input, wherein the modified neural network can be identified using the trigger input, the method comprising: obtaining a trained neural network; determining an identification neuron for the trained neural network, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network; and generating, using the modified neural network, the trigger input such that; when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; wherein the identification neuron is such that, when the identification neuronis triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network. It may be that triggering the identification neuron comprises activating the identification neuron. Thus, it may be that, in the absence of the trigger input, the identification neuron remains relatively inactive. It may be that the identification neuron responds to the presence of the trigger input in an input to the modified neural network by becoming relatively active. It may be that the triggering the identification neuron comprises deactivating the identification neuron. Thus, it may be that, in the absence of the trigger input, the identification neuron is relatively active. It may be that the identification neuron responds to the presence of the trigger input in an input to the modified neural network by becoming relatively inactive. It will be appreciated that, in this context, the terms “relatively active” and “relatively inactive” are used to indicate that the identification neuron is more active in one condition than the other. Thus, for example, the identification neuron becoming relatively active after having been relatively inactive, simply indicates an increase in activity of the identification neuron. Thus, it may be that the triggering of the identification neuron comprises stimulating a response by the identification neuron. It may be that the identification neuron has an inoperative condition (for example, in which the identification neuron is not significantly affecting operation of the modified neural network) and an operative condition (for example, in which the identification neuron is affecting operation of the modified neural network). It may be that the triggering of the identification neuron comprises causing the identification neuron to transition from the inoperative condition to the operative condition. It will be appreciated that an identification neuron may be either active (i.e. generating an output) or inactive (i.e. not generating a significant output) when in the operative condition. It will be appreciated that references to a “response” of a neuron refer to a change in state of that neuron. For example, it may be that an example neuron ordinarily generates a first output. The “response” of that example neuron to a given input refers to a change in the output of that neuron in response to the given input. Similarly, it will be appreciated that references to the “sensitivity” of a neuron to a particular input refer to the propensity of that neuron to respond to that particular input. It may be that the one or more of the associated weights and biases of thedetermined identification neuron are zero values. It will be appreciated that, where,for example, the associated weights and biases include only biases having zero values, this is effectively equivalent to the determined identification neuron having associatedweights only. It may be that determining the identification neuron comprisesdetermining at least one of the associated weights of the determined identification neuron. It may be that determining the identification neuron comprises determining at least one of the associated biases of the determined identification neuron. It may be that one or more further weights or biases of the determined identification neuron arepredetermined. It may be that determining the identification neuron comprisesdetermining the associated location in the modified neural network of the identification neuron. It may be that determining the associated location comprises identifying a neuron in the trained neural network which is to be replaced by the identification neuron. It may be that determining the associated location comprises identifying one or more neurons in the trained neural network to which the identification neuron is to be connected. It will be appreciated that, in this context, a first neuron is connected to a second neuron if an output of the one of the first neuronand the second neuron is received at the other of the first neuron and the secondneuron as an input. It may be that the method comprises determining one or more (for example, all) weights and / or biases of the identification neuron. It may be that one or more (for example, all) weights or biases of the determined identification neuron are randomly selected. Thus, it may be that determining the one or more weights and / or biases of the identification neuron comprises randomly selecting the one or more weights and / or biases.It may be that the method comprises evaluating the one or more randomlyselected weights and / or biases. It may be that the evaluating of the one or more randomly selected weights and / or biases comprises determining whether the one ormore randomly selected weights and / or biases will cause the identification neuron tobe triggered by a natural input to the modified neural network. It will be appreciatedin this context that a “natural” input is one which is of the type that the trained neuralnetwork would likely be presented with when deployed and in operation. It has beenobserved that clouds of feature vectors in artificial intelligence models (for example,computer vision models) tend to be similar in shape in high-dimensional space. Thus,it may be that feature vectors of a natural input to the modified neural network haveone or more principal directions of variation (for example, within the cloud of featurevectors of the artificial intelligence model). Thus, it is possible to generate an input tothe modified neural network that has feature vectors having different directions to theone or more principal directions associated with a natural input. It may be that one ormore weights and / or biases of the identification neuron are configured to cause the identification neuron to be triggered by inputs having feature vectors with differentdirections to the one or more principal directions associated with a natural input.It may be that the modified neural network connects one or more inputs of the modified neural network to the inputs of the identification neuron via one or more other neurons in the modified neural network. It will be appreciated that theconnection of an input of the modified neural network to an input of the identificationneuron can be defined by a sub-graph (for example, a reduced section of a graphdefined by the modified neural network). It may be that determining whether the one or more randomly selected weights and / or biases will cause the identification neuron to respond to a natural input comprises determining and evaluating one or more such sub-graphs. It may be that determining whether the one or more randomly selected weights and / or biases will cause the identification neuron to respond to a natural input comprises determining a function that maps inputs to the modified neural network to an input of the identification neuron. It may be that determining whether the one or more randomly selected weights and / or biases will cause the identification neuron torespond to a natural input comprises evaluating that function. Alternatively, it may bethat determining whether the one or more randomly selected weights and / or biases will cause the identification neuron to respond to a natural input comprises presenting the modified neural network with a plurality of inputs (for example, natural inputs)and evaluating the resulting outputs of the modified neural network.The method may further comprise, in response to the evaluating indicating that the one or more randomly selected weights and / or biases will cause the identification neuron to respond to a natural input, determining (for example, by random selection) one or more new weights and / or biases for the identification neuron. It may be that the method comprises repeating the random selection and the evaluating of the resulting one or more weights and / or biases until the evaluating indicates that the one or more weights and / or biases are unlikely to cause the identification neuron to respond to a natural input. It may be that the trigger input has one or more characteristics that are not ordinarily present in inputs to the trained neural network. It may be that the one or more constraints operate to increase the chances that the random selection of the one or more weights or biases yields an identification neuron that is sensitive to the one or more characteristics. Similarly, it may be that inputs to the trained neural network not incorporating the trigger input have one or more further characteristics in common. It may be that the one or more constraints operate to minimise the chances that the random selection of the one or more weights or biases yields an identification neuronthat is sensitive to the one or more further characteristics.It may be that generating the trigger input comprises obtaining a base triggerinput. It may be that the base trigger input is configured to at least partially trigger theidentification neuron. It may be that generating the trigger input comprises optimising the base trigger input to generate an optimised trigger input. It may be that the identification neuron has a greater response to the optimised trigger input than to thebase trigger input. It may be that optimising the base trigger input comprisesiteratively generating one or more candidate trigger inputs. In such cases, it may be that optimising the base trigger input comprises evaluating a response of the identification neuron to each of the one or more candidate trigger inputs. It may bethat optimising the base trigger input comprises, on the basis of the one or morecandidate trigger inputs and the evaluated responses, generating one or more further candidate trigger inputs. It may be that the generated one or more further candidate trigger inputs elicit trigger the identification neuron to a greater degree (for example, by eliciting an increased response from the identification neuron) than the one or more candidate trigger inputs. It may be that evaluating a response of the identificationneuron comprises evaluating an output of the modified neural network. Thus, it maybe that evaluating a candidate trigger input comprises presenting the modified neural network with the candidate trigger input and assessing the resulting output of themodified neural network. It may be that evaluating a candidate trigger input comprisesassessing a function that quantifies the response of the identification neuron to thecandidate trigger input. It may be that the function indicates a distance between theresponse of the modified neural network to the candidate trigger input and a desiredresponse to the trigger input. Thus, it may be that the function is formulated such that a lower result indicates greater activation of the identification neuron. It may be that optimising the base trigger input comprises performing a gradient descentoptimisation (for example, to minimise the result of the function).It may be that the trained neural network is configured to, in response to being presented with a given input not comprising the trigger input, generate a corresponding given output. It may be that the modified neural network is configured to, in response to being presented with the given input, generate the corresponding given output. It may be that the modified neural network is configured to, in response to being presented with the given input modified to incorporate the trigger input,produce a different output to the given output. Thus, it may be that the modifiedneural network is configured generate, in response to having been presented a given input, one of two outputs in dependence on whether the identification neuron has been triggered (i.e. depending on whether the given input has been modified to incorporate the trigger input). It may be that the trained neural network is configured such that, when the trained neural network is presented with a given input dataset (for example, a validation dataset) not including the trigger input, it produces an associated set of outputs. It may be that the modified neural network is configured such that, when the modified neural network is presented with the given input dataset, it also produces the same associated set of outputs. It may be that the desired modification to the output of the modified neural network results in a modified output of the modified neural network. It may be that, when the identification neuron is not triggered by an input to the modified neural network, the output of the modified neural network is not significantly modified. Thus, it may be that, when the identification neuron is not triggered by an input to themodified neural network, the modified neural network generates outputs matchingthose of the trained neural network.It may be that the method further comprises recording the modified output ofthe modified neural network (i.e. that arising due to triggering of the identificationneuron). The method may further comprise, subsequent to the recording, presenting afurther neural network with the trigger input, such that the further neural network generates a further output. It may be that the method further comprises comparing the further output to the recorded modified output to determine whether the further neuralnetwork is based on the modified neural network. In this context, it will beappreciated that a further neural network can be said to be based on the modified neural network if it is the same as or an altered version of the modified neural network. Thus, it may be that the further neural network comprises an instance of the modified neural network. It may be that the further neural network comprises analtered version of the modified neural network. In such cases, it may be that thealtered version of the modified neural network has been produced by performingfurther training of the modified neural network. It may be that such further training was performed after the implanting of the identification neuron. It may be that an output of the implanted identification neuron is connected toone or more further neurons in the modified neural network. It may be that themodified neural network is configured such that one or more further neurons dependon the output of the identification neuron. It may be that the one or more furtherneurons are not immediately adjacent to the identification neuron. Thus, it may be thatthe connection of the identification neuron to the one or more further neurons is via at least one other neuron in the modified neural network. It may be that the method comprises modifying a weight and / or bias of the at least one other neuron to ensure that the output of the identification neuron propagates through the neural network to the one or more further neurons. It may be that the connection of the implantedidentification neuron to the one or more further neurons is such that triggering theimplanted identification neuron activates or deactivates a portion of the modified neural network. Hence, the one or more further neurons may be referred to as “disruption neurons”. Deactivating a portion of the modified neural network may comprise causing at least one neuron in the portion to become unresponsive to changes in inputs to the modified neural network. Deactivating a portion of the modified neural network may comprise causing at least one neuron in the portion to become respond differently to changes in inputs to the modified neural network than when the portion is active. Thus, it may be that the modified neural network is ordinarily configured (i.e. when the portion is active) to generate a given output in response to a given input. It may be that disabling the portion of the modified neural network causes the modified neural network to generate, in response to the given input, an output that is different to the given output. It may be that the activation or deactivation of the portion of the modified neural network causes one or morefunctions of the modified neural network to be enabled or disabled. Thus, themodified neural network may be said to be “function-locked”. For example, it may be that the modified neural network is configured such that one or more functions of themodified neural network are disabled unless an input to the modified neural networkcomprises the trigger input. It may be that the implanting comprises adding an additional neuron to thetrained neural network. It may be that the implanting comprises adding multipleadditional neurons to the trained neural network. Thus, it may be that the modified neural network comprises one or more neurons that were not present in the trainedneural network. It may be that the implanting comprises modifying one or moreweights or biases of an existing neuron in the trained neural network. Thus, it may bethat the modified neural network has the same number of neurons as the trained neural network. It may be that a plurality of neurons (for example, more than 90% of the neurons) in the modified neural network are unaltered compared to the trained neural network. Thus, it may be that a plurality of neurons (for example, more than 90% of the neurons) in the modified neural network have weights and biases that are the same are those of the corresponding neurons in the trained neural network. It may be that only one neuron differs between the modified neural network and the trained neural network. Thus, it may be that all but one neuron in the modified neural network is unaltered compared to the trained neural network. It may be that the associated location of the implanted neuron is in a hiddenlayer of the modified neural network. It will be appreciated that a hidden layer of aneural network is one which is not directly accessible as an input or output of theneural network (i.e. an intermediate layer of the neural network between its input andoutput layers). It may be that the method comprises identifying a further neural network. Itmay be that identifying the further neural network comprises obtaining operationalinput data and embedding the trigger input into the obtained operational input data. It may be that identifying the further neural network comprises providing the operational input data having the embedded trigger input to the further neural network and evaluating the resulting output. It may be that evaluating the resulting output comprises comparing the output to the known desired modification to the output of the modified neural network arising from activation of the identification neuron. Thus, it may be that the desired modification to the output of the modified neural network can be used to identify the modified neural network. For example, where a further neural network is presented with the trigger input and the resulting output reflects the desired modification to the output of the modified neural network arising from activation of the identification neuron, it can be inferred that the further neural network is at least based on the modified neural network. The modified neural network can be referred to as a “stained” or “watermarked” version of the trained neural network, wherein the implanting of the identification neuron constitutes applying a “stain” or “watermark” to the neural network. It may be that the method comprises operating the modified neural network. Itmay be (for example, where the modified neural network is configured to perform functionality locking) that operating the modified neural network comprises obtaining operational input data and embedding the trigger input into the obtained operational input data (for example, to allow the input data to enable one or more function-locked functions of the modified neural network). It may be that the method further comprises performing (for example, subsequent to the implanting) further training of the modified neural network. In suchcases, it may be that the weights and biases of identification neuron persist throughthe further training. As discussed above, it may be that determining the identificationneuron comprises identifying the associated location in the modified neural network.It may be that the identifying of the associated location comprises identifying aneuron that is relatively inactive when the trained neural network is presented with a natural input. It will be appreciated in this context that a “natural” input is one which is of the type that the trained neural network would likely be presented with when deployed and in operation. It may be that identifying a neuron that is relatively inactive comprises identifying a neuron that, when the trained neural network is presented with a natural input, has an output that is below a predetermined threshold. Selecting a neuron that is relatively inactive when the trained neural network is presented with a natural input to act as the identification neuron can reduce the likelihood of the identification neuron being affected by further training of the modified neural network. Because the selected neuron is relatively unresponsive to natural inputs, further training of the modified neural network using natural inputs is unlikely to activate the selected neuron, reducing the likelihood of the further training process altering the weights and / or biases of the implanted identification neuron. Itmay be that the weights and biases of identification neuron persist such that, when theidentification neuron is triggered by an input to the further trained neural network, it causes the desired modification to the output of the further trained neural network. It may be that the trained neural network is configured to perform one or morecomputer vision functions in respect of an input image. The one or more computervision functions may comprise one or more of image recognition, object classification, and object detection. It may be that the trained neural network isconfigured to perform the one or more computer vision functions by virtue of itstraining. Thus, the method may comprise operating the modified machine learningagent to perform the one or more computer vision functions in respect of an inputimage. Thus, it may be that the trigger input comprises at least a portion of the inputimage. It may be that the trigger input forms a portion (for example, a patch) of theinput image). Thus, it may be that a portion of the input image has been altered to incorporate the trigger input. In such cases, it may be that a further portion of the input image is unaltered by the inclusion of the trigger input. It may be that the size (for example, by number of pixels) of the portion of the input image is less than 25%, less than 15%, less than 10%, less than 5%, or less than 1% of the size of the inputimage. It may be that the trained neural network is configured to generate text data inresponse to input text prompts. Thus, it may be that the trigger input comprises a textprompt. It will be appreciated that the present invention is not limited to suchapplications, but rather is applicable to neural networks and to machine learning agents in general. It may be that the method comprises determining a plurality of identification neurons for the trained neural network. In such cases, it may be that each of the identification neuron in the plurality has a respective one or more associated weights and biases and a respective associated location in the modified neural network. It may be that the associated weights and biases of a first neuron in the plurality differ fromthose of a second of neuron in the plurality. It may be that the associated weights and biases of a first neuron in the plurality differ from those of all of the neurons in the plurality. Where the method comprises determining a plurality of identification neurons, it may be that the method further comprises implanting the plurality of identification neurons in their respective associated locations in the modified neural network. It will be appreciated that the characteristics described above with respect to a single identification neuron may be present in each of the plurality of identification neurons. Where the method comprises determining and implanting a plurality of identification neurons, it may be that the functionality described above enabled by the identification neuron is enabled by the plurality of identification neurons collectively. It may be that the modified neural network comprises a long short-term memory network. In cases where the modified neural network implements a function- lock, it may be that the long short-term memory network operates to suppress the function-lock for a number of uses of the modified neural network. For example, itmay be that the long short-term memory network operates to allow use of the fullfunctionality of the modified neural network for a trial period before subsequently requiring that inputs to modified neural network incorporate the trigger input to unlock the full functionality of the modified neural network. According to a second aspect of the invention there is provided a computer program product comprising instructions which, when the program is executed by a computing device, cause the computing device to perform a method according to the first aspect. According to a third aspect of the invention there is provided a computer-implemented method of operating a modified neural network, the neural network having been modified by: determining an identification neuron for the trained neural network, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; and implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network, the method comprising: providing to the modified neural network, an input comprising a trigger input, the trigger input having been generated such that: when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; wherein the identification neuron is such that, when the identification neuronis triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network, the desired modification being such that the modified neural network can be identified using the trigger input. According to a fourth aspect of the invention there is provided a computerprogram product comprising instructions which, when the program is executed by a computing device, cause the computing device to perform a method according to the third aspect. According to a fifth aspect of the invention there is provided an apparatuscomprising a modified neural network, wherein: the neural network has been modified by: determining an identification neuron for the trained neural network, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; and implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network; the modified neural network is configured to receive a trigger input, the trigger input having been generated such that: when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; and the identification neuron is such that, when the identification neuron istriggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network, the desired modification being such that the modified neural network can be identified using the trigger input. It will of course be appreciated that features described in relation to one aspect of the present invention may be incorporated into other aspects of the present invention. For example, the method of the invention may incorporate any of thefeatures described with reference to the apparatus of the invention and vice versa.Description of the Drawings Embodiments of the present invention will now be described by way of example only with reference to the accompanying schematic drawings of which: Figure 1 shows a flow chart illustrating the steps of a method according to a firstembodiment of the invention; andFigure 2 shows a schematic view of an apparatus according to a secondembodiment of the invention. Detailed Description Figure 1 shows a flow chart illustrating the steps of a computer-implementedmethod 100 of generating a modified neural network and a trigger input according toa first embodiment of the invention. The modified neural network and the trigger input are such that the modified neural network can be identified using the trigger input. A first step, represented by item 101, of the method 100 comprises obtaining atrained neural network. In this example embodiment, the trained neural network isconfigured to perform object recognition of one or more objects is an input image. Thus, in this example embodiment, the trained neural network is configured to receive an image as an input and to generate a plurality of scores indicating a measure of confidence of the neural network that an object in the image belongs to each of a respective plurality of categories. It will be appreciated that in other embodiments the trained neural network may be configured to perform alternative functions. For example, in other embodiments the trained neural network may, alternatively or additionally, be configured to perform one or more other computer vision functions (for example, image recognition or object detection). Similarly, in other embodiments the trained neural network may be to generate text data in response to input text prompts. A second step, represented by item 103, of the method 100 comprises determining an identification neuron for the trained neural network. The identification neuron has an associated location in the modified neuralnetwork (i.e. a location in the modified neural network where the identificationneuron is to be implanted). In this example embodiment, the identification neuron isto be implanted as an additional neuron not present in the trained neural network. Thus, the associated location in the modified neural network comprises the layer of the modified neural network in which the identification neuron is to be implanted and what other nodes in the modified neural network the identification neuron is to be connected to. In alternative embodiments, the identification neuron is implanted by replacing an existing neuron in the trained neural network. In such cases, theassociated location comprises an indication of which neuron in the trained neuralnetwork is to be replaced. In this example embodiment, the associated location of theimplanted neuron is in a hidden layer of the modified neural network. The identification neuron also has one or more associated weights and biases. The weights and biases of the determined identification neuron are such that the identification neuron is relatively unresponsive to the content of natural inputs to the trained neural network. It will be appreciated in this context that a “natural” input is one which is of the type that the trained neural network would likely be presentedwith when deployed and in operation. Thus, the weights and biases of the determinedidentification neuron are such that the determined identification neuron is not significantly stimulated by natural inputs to the modified neural network. The identification neuron is determined such that, when the identification neuron (once implanted into the trained neural network to produce the modified neural network as detailed below) is triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network. In this example embodiments, the determined identification neuron is ordinarily (i.e. when the modified neural network is not presented with the trigger input) inactive. Thus, in this example embodiment, triggering the comprises activating theidentification neuron. The identification neuron is therefore relatively inactive in theabsence of the trigger input in an input to the modified neural network. When the trigger input is present in an input to the modified neural network, the identification neuron responds by becoming relatively active. It will be appreciated that in otherembodiments, the determined identification neuron is ordinarily (i.e. when themodified neural network is not presented with the trigger input) active. In such cases,identification neuron is therefore relatively active in the absence of the trigger input inan input to the modified neural network and, when the trigger input is present in an input to the modified neural network, the identification neuron responds by becomingrelatively inactive. The desired modification to the output of the modified neuralnetwork (i.e. that arising from triggering of the identification neuron) results in the modified neural network generating a modified output. When the identification neuron is not triggered by an input to the modified neural network, the output of themodified neural network is not significantly modified (i.e. compared to that of thetrained neural network). It will be appreciated that the trained neural network can be said to be configured to, in response to being presented with a given input, generate a corresponding given output. Where the given input does not comprise the trigger input, the determined identification neuron is such that modified neural network is configured to, in response to being presented with the given input, generate thecorresponding given output. Thus, the determined identification neuron is such that,for a given input not incorporating the trigger input, the trained neural network andthe modified neural network produce substantially the same output. The determinedidentification neuron is also such that the modified neural network is configured to, inresponse to being presented with the given input modified to incorporate the triggerinput, produce a different output to the given output. Thus, the determinedidentification neuron is such that, for a given input incorporating the trigger input, the trained neural network and the modified neural network produce differing outputs. In this example embodiment, the determined identification neuron hasrandomly selected weights and biases. In particular, in this example embodiment, theweights and biases of the identification neuron were determined by repeatedlyrandomly selecting the weights and biases and evaluating whether those randomlyselected weights and biases would cause the identification neuron to be triggered by a natural input to the modified neural network. The random selection and evaluation of weights and biases was repeated until the evaluation indicated that the selected weights and biases would not cause the identification neuron to be triggered by a natural input to the modified neural network. The evaluation of the weights and biases was performed by evaluating a sub-graph mapping the inputs of the modified neuralnetwork to the input of the identification neuron. It will be appreciated that, in otherembodiments, weights and / or biases of the identification neuron may be selected in other ways. For example, it may be that one or more of the weights and / or biases are predetermined. In some embodiments, it may be that only a subset of the weights and biases of the identification neuron are randomly selected. In this example embodiment, the modified neural network is further configured to implement a function-lock. The modified neural network is configured such that one or more further neurons depend on the output of the identification neuron. An output of the determined identification neuron is connected to one or morefurther neurons (i.e. disruption neurons) in the modified neural network such that thetriggering the determined identification neuron activates a portion of the modified neural network. The one or more further neurons are not immediately adjacent to the identification neuron, such that the connection of the identification neuron to the disruption neurons is via at least one other neuron in the modified neural network. Therefore, the method further comprises modifying a weight and / or bias of the at least one other neuron to ensure that the output of the identification neuron propagates through the neural network to the disruption neurons. It may be that the connection of the implanted identification neuron to the one or more further neurons is such that triggering the implanted identification neuron activates or deactivates a portion of the modified neural network. The activation of the portion of the modified neural network causes one or more functions of the modified neural network to be enabled. Thus, the determined identification neuron operates to disable the one or more functions of the modified neural network unless the input to the modified neural network comprises the trigger input. In this example embodiment, the determined identification neuron operates to disable the one or more functions by causing at least one of the connected one or more further neurons to become unresponsive to changes in inputs to the modified neural network. However, it will be appreciated that a function-lock may also be implemented in other ways. For example, a function-lock may alternatively be implemented such that the determined identification neuron operates to disable theone or more functions by causing at least one of the connected one or more furtherneurons to respond differently to inputs than when identification neuron has not been triggered. It will further be appreciated that in other embodiments, the function-lock may alternatively be configured such that triggering the determined identification neuron deactivates a portion of the modified neural network, causing one or more functions of the modified neural network to be disabled. A third step, represented by item 105, of the method 100 comprises implanting the identification neuron into the trained neural network in the associated location toproduce the modified neural network. As discussed above, in this exampleembodiment, the implanting comprises adding an additional neuron to the trained neural network. Thus, the modified neural network comprises an additional neuron (i.e. the identification neuron) that is not present in the trained neural network. Otherwise, the modified neural network is unchanged compared to the trained neural network (i.e. the weights and biases of neurons in the modified neural network other than the identification neuron are the same as those of the corresponding neurons inthe trained neural network). In alternative embodiments, the implanting may comprisemodifying one or more weights or biases of an existing neuron in the trained neural network. A fourth step, represented by item 107, of the method 100 comprises generating, using the modified neural network, the trigger input. The trigger input isgenerated such that when the trigger input is present in an input to the modified neuralnetwork, the identification neuron is triggered. In this example embodiment, the trigger input is further generated such that when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered. However, it will be appreciated that, in other embodiments, the trigger input and identification neuron may be such that the identification neuron is not triggered when the trigger input is present in an input to the modified neural network and theidentification neuron is triggered when the trigger input is not present in an input tothe modified neural network. In this example embodiment, generating the trigger input comprises obtaining a base trigger input and optimising the base trigger input to generate an optimised trigger input. The base trigger input is configured to at least partially trigger the identification neuron. The optimising of the trigger input is such that the identification neuron has a greater response to the optimised trigger input than to the base triggerinput. Optimising the base trigger input comprises iteratively generating one or morecandidate trigger inputs. The modified neural network is presented with each of the one or more candidate trigger inputs and the output of the modified neural network isevaluated to determine the responses of the identification neuron to the candidatetrigger inputs. Based on the one or more candidate trigger inputs and the evaluated responses,one or more further candidate trigger inputs are then generated. The generation ofcandidate trigger images and evaluation of the associated responses of the identification neuron to those candidate trigger inputs is repeated iteratively, with the aim that each generation candidate trigger inputs elicits an increased response from the identification neuron compared to the preceding generation, to generate the optimised trigger input. As discussed above, the weights and biases of the determined identification neuron are such that the determined identification neuron is not significantly stimulated by natural inputs to the modified neural network. By contrast, theoptimised trigger input is configured to stimulate the identification neuron. Thus, theweights and biases of the determined identification neuron are such that the trigger input elicits a response from the identification neuron. The optimised trigger input therefore constitutes an input that is highly unlikely to appear in natural inputs to the modified neural network (and so is highly unlikely to be encountered when the modified neural network is deployed and in operation, unless it has been deliberately superimposed onto a given input). It will be appreciated that, whilst in this example embodiment the response of the identification neuron is evaluated by assessing the output of the modified neural network, in other embodiments the response of the identification neuron may be evaluated in other ways (for example, by directly monitoring an activation of identification neuron). Similarly, whilst in this example embodiment the optimising of the trigger input is performed using a gradient descent algorithm, it will be appreciated that the optimisation may alternatively be performed in other way and / or by use of other optimisation algorithms or techniques. An optional fifth step, represented by item 109, of the method 100 comprisesrecording the modified output of the modified neural network (i.e. the output arisingwhen the identification node is triggered). An optional sixth step, represented by item 111, of the method 100 comprises (i) obtaining operational input data and (ii) embedding the trigger input into theobtained operational input data. Where the identification neuron and the modifiedneural network are configured to implement a function-lock, the step of embedding of the trigger input into the obtained operational input data can be used to control use of the modified neural network. For example, it may be that one or more functions of the modified neural network are disabled unless the trigger input is present in the input to the neural network. Where the modified neural network comprises a long short-term memory network, the function-lock can be suppressed for a number of uses of the modified neural network. An optional seventh step, represented by item 113, of the method 100comprises subsequent to the recording, (i) presenting a further neural network with thetrigger input, such that the further neural network generates a further output (ii) comparing the further output to the recorded modified output to determine whether the further neural network is derived from the modified neural network. Thus, the method may comprise identifying the further neural network. As discussed previously, the determined identification neuron is configured such that it is highlyunlikely to respond to natural inputs. Such a neuron is highly unlikely to arise duringtraining of a machine learning agent to operate on natural inputs. Therefore, the presence of the identification neuron in the further neural network (as indicated by the incidence of the desired modification in the further output) is indicative that the further neural network is based on (for example, is substantially identical to or is derived from) the modified neural network. An optional eighth step, represented by item 115, of the method 100 comprises subsequent to the implanting, performing further training of the modified neural network. The determining and implanting of the identification neuron are such that theweights and biases of identification neuron persist through the further training. Thus,when the identification neuron is triggered by an input to the further trained neural network, it causes the desired modification to the output of the further trained neural network. Provided below is a mathematical description of an example method accordingto embodiments of the present invention. In the following description: a function isdeclared using the syntax ^ ∶ ^ → ^ where A denotes the set of permissible inputs to fand outputs produced by f are elements of the set B; R denotes real numbers; for apositive integer d, the notation ^^ denotes the space of vectors with d real-valuedcomponents; tensors over R are interpreted as grids of real numbers in any dimension(a 0-tensor is a real number, a 1-tensor is a vector, a 2-tensor is a matrix - a stack ofvectors which all have a common shape; a 3-tensor is a stack of matrices which allhave a common shape, and so on; for a set A and a set B, their Cartesian product isdenoted by ^ × ^, defined as the set of all pairs where the first component is anelement of A and the second component is an element of B; for a set A and a set B, thenotation ^ ⊆ ^ indicates that A is either equal to B or it is a subset of B; and for a setA and a function ^ ∶ ^ → ^, the notation arg ^^^^∈^^(^) denotes the set of elementsin A on which f attains its minimum value (i.e. for any x in this set and any x’ not inthis set, f(x’) > f(x).The modified neural network constitutes a model N taking images as inputs,where S denotes the space of the images (i.e. of the inputs to the model). Theidentification neuron has weights w and bias b. The mapping of an input of N to theinput of a layer f of the network is represented by ^ ∶ ^ → ^^. This corresponds tosequentially evaluating the nodes / edges of the model, starting from the input, until allinputs to f have been computed (but without evaluating f itself). The base trigger inputτ to be optimised can be any image. A function ^: ^ → ^ is used to evaluate the qualityof a candidate trigger input. The function is formulated such that a lower resultconstitutes a better trigger input. A stain detector distribution D is defined on Rd. Thisis typically taken as the uniform distribution on the surface of the d-dimensionalsphere with radius 1. Well-known standard algorithms exist for sampling from thisdistribution. A desired trigger response is defined as Δ ∈ ^ and a desired non-triggerresponse is defined as δ ∈ ^. Thus, when the trigger image is used as an input, theidentification neuron will output value Δ. Otherwise (on average), it will output δ. Afinite set of validation data is defined as ^ ^. The validation data comprises a set ofimages not containing the trigger input. In this example embodiment, the method comprises: (i) sampling a vector ω from the distribution D,(ii) use a gradient descent optimisation method to iteratively improve the basetrigger image τ into the final trigger image ^∗ ∈ ^, with the minimisationobjective being ^(^) = ^(^) − 〈^, ^(^)〉,(iii) define ^, ^ ∈ ^ such that wherein if V is empty ^ = 0 is chosen,(iv) replace the weight vector w with ^^ and the bias value b with ^ (typically thiswill mean replacing one row of a weight matrix and one entry in a bias vector), such that the response of the neuron to input image ^∗will be Δ and its average response to other images will be δ, (v) return the resulting model as the modified version of N, and(vi) evaluate the output of the identification neuron when the trigger input ^∗ isused as the input (with high probability this value will be close to Δ, while for any other input the value will be close to δ, which should be far from Δ). The method may be further continued to develop the identification neuron into a function lock, for example as described below. Where the identification neuron comprises a convolutional neuron, it may be that the convolutional neuron has a relatively small receptive field. Thus, it may be that, for each position of the convolution, the output of the neuron doesn’t depend on a significant number of pixels from the original image. The modified neural network further comprises another neuron (i.e. a disrupter neuron) which depends on theoutput from the identification neuron, to be used to produce disruption when theactivation trigger is (in this example) not present. Suppose that the disrupter neuron evaluates a linear combination of its inputs, and one of those inputs depends on theoutput of the identification neuron (i.e. a detector signal). Thus, it may be that thedisrupter neuron has d1 inputs, and a weight vector ^ ∈ ^^^ and bias ^ ∈ ^ and for aninput vector ^ ∈ ^^^ it computes the output ^ ∙ ^ + ^. A distribution B is defined onthe real numbers R. The function lock is implemented by (in addition to the steps above): (vii) finding the chain of network layers connecting the output of the identificationneuron to the disrupter neuron,(viii) modify these neurons such that they pass the detector signal from the detectorto the disrupter without affecting other signals within the model (for example,for a neuron which outputs a weighted linear combination of its inputs, replace all weights with zero, except for the weight of the detector signal),(ix) let ^ ∈ ^ be the strength of the detector signal at the input to the disrupterneuron when the trigger is presented to the modified neural network,(x) suppose that only the component i of the input to the disrupter neuron dependson the detector signal, (xi) sample a new bias value β from the distribution B, and(xii) replace component i of ^ with the value and replace the bias value ^ with^. It will be appreciated that other embodiments of the method also exist. For example, a more general example method is described below. The modified neural network constitutes a model N taking inputs in a set S.The model N may be expressed via a 'computational graph' (for example, a directedacyclic graph of functions) where: (i) each node represents evaluating a function withinputs which are tensors over R and an output which is a tensor over R, and (ii) eachedge represents passing a value from the output of one operation to the input of another. There exists an operation within the graph which evaluates a linear functionalof its input, which is denoted as ^: ^^ → ^. Let ^: ^ → ^^ represent the mapping ofan input N to the input of f. This function can be constructed by finding thenodes / edges of the graph which are 'upstream' of f, and building the correspondingsub-model. The base trigger input τ to be optimised can be any valid input to N. Afunction ^: ^ → ^ is used to evaluate the quality of a candidate trigger input. Thefunction is formulated such that a lower result constitutes a better trigger input. Adetector distribution D is defined on Rd. A map is defined as ^⋅,⋅^ ∶ ^^ × ^^ → ^. Adetector response mapping function is defined as ^: ^ → ^.In this general example, the method comprises: (i) sampling a vector ω from the distribution D,(ii) defining a trigger input ^∗ ∈ ^ as a solution to the problem:^∗ ∈ arg m^∈i^n^(^) where it is defined that ^(^) = ^(^) − 〈^, ^(^)〉,(iii) defining the function ^: ^^ → ^ such that ^(^) = ^(〈^, ^〉) for any ^ ∈ ^^,and (iv) constructing the stained model ^ formed of the same graph as N but with thenode corresponding to the operation f replaced with a node evaluating g.This general example method may also be further continued to develop the identification neuron into a function lock, for example as described below. ∗The stained model ^ as and trigger ^ are generated as described above bystaining node f of N. A set of disruption nodes {ℎ : ^ → ^^^}^^ ^^^ in the model N are‘downstream’ of f (i.e. having inputs depending on the output of f), to be used toproduce the disruption. Disrupter distributions are defined on ^^^ for each of 1 ^ ≤ ^.The function lock is implemented by (in addition to the steps above): (v) defining that ^ = ^,^(vi) for each of 1 ≤ ^ ≤ ^:a. define the model ^ by replacing a sequence of existing nodes and edges^of the graph of to propagate the output signal from f to ℎ (the ^to ℎ should not depend strongly on the outputs of other parts of the model^and the inputs to other parts of the model should not depend on the output from f),b. define ^^ ∶ ^ → ^ to be the mapping representing the combined action ofall these nodes and edges linking f and ℎ^ (therefore for an input z to themodel ^^, the input to node ℎ^can be expressed as ^^(^^^(^)^), c. sample a vector ^^ from the distribution ^^,d. define ℎ^^ ∶ ^ → ^^^ such that any ^ ∈ ^, such that e. construct the model ^^ from the model ^^ by replacing node ℎ^ in thecomputational graph with node ℎ^^, and (vii) return the model ^^ as the function-locked version of N.Figure 2 shows a schematic view of an apparatus 200 according to a second embodiment of the invention. The apparatus 200 comprises a modified neural network 201. The modified neural network 201 is configured to receive an input 203 and, on the basis of that input 203 and a plurality of weights and biases of neurons making up the modified neural network 201, to generate an output 205. The modified neural network 201 is configured to operate as described above in respect of the method 100. For example, the modified neural network 201 the neural network has been modified by implanting the identification neuron into a trained neural network to produce the modified neural network 201. The identification neuron has one or more associated weights and biasesand an associated location in the modified neural network. The identification neuronhas been implanted into the trained neural network at the associated location toproduce the modified neural network 201. Furthermore, the modified neural network201 is configured to receive a trigger input. That trigger input is such that, when the trigger input is present in an input to the modified neural network, the identification neuron is triggered, and, when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered. The identification neuron is also such that, when the identification neuron is triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network, the desired modification being such that the modified neural network can be identified using the trigger input. Any or all of the optional features of the method 100 described above may also be implemented by the apparatus 200. The apparatus 200 further comprises a processor 207 and an associated memory 209. The processor 207 may be configured to implement (for example, by executing instructions stored in the memory 209) the functions of the modified neural network 201 described above. The processor 207 may be configured to cause the apparatus 200 (for example, by executing instructions stored in the memory 209) to perform the method 100 described above. Whilst the present invention has been described and illustrated with reference to particular embodiments, it will be appreciated by those of ordinary skill in the art that the invention lends itself to many different variations not specifically illustrated herein. By way of example only, certain possible variations will now be described. “The Feasibility and Inevitability of Stealth Attacks” by Ivan Y. Tyukin et al(arXiv:2106.13997) describes techniques for determining and implanting a compromised neuron into a neural network, but for the purpose of compromising a third party neural network. The present invention is a development of thesetechniques, built on the recognition that such techniques can expanded and applied toenable different new functionality including the identification and / or function lockingof a neural network. Whilst the example embodiments described above implement a function-lock, it will be appreciated that this is need not necessarily be so. Some other embodiments of the invention do not implement a function-lock. Instead such embodiments may operate merely to enable identification of the modified neural network (for example, by “staining” or “watermarking” of the modified neural network). Whilst the example embodiments described above are described as having only a single identification neuron, it will be appreciated that the method 100 and apparatus 200 described above may incorporate multiple identification neurons. In such cases, it may be that a first of the multiple identification neurons is associated with a first location in the modified neural network and a second of the multiple identification neurons is associated with a different second location in the modified neural network. It may be that the first of the multiple identification neurons is implanted by adding an additional neuron to the trained neural network. It may be that a second of the multiple identification neurons is implanted by replacing the weights and / or biases of an existing neuron in the trained neural network. Whilst in the example embodiments described above, the weights and biases of the determined identification neuron have been randomly selected, it will be appreciated that other ways of selecting weights and biases may alternatively be used. For example, in other embodiments, it may be that the weights and biases are predetermined. It will be appreciated that the apparatus 200 may comprise one or more processors and / or memory. As mentioned above, the apparatus 200 may comprise a processor 207 and associated memory 209.The processor 207 and associated memory 209 may be configured to perform one or more of the above-described functions of the apparatus 200. Each device, module, component, machine or function as described inrelation to any of the examples described herein (for example, the modified neuralnetwork 201) may similarly comprise a processor or may be comprised in apparatus comprising a processor. One or more aspects of the embodiments described herein comprise processes performed by apparatus. In some examples, the apparatus comprises one or more processors configured to carry out these processes. In thisregard, embodiments may be implemented at least in part by computer software storedin (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware). Embodiments also include computer programs, particularly computer programs on or in a carrier, adapted for putting the above-described embodiments into practice. The program may be in the form of non-transitory source code, object code, or in any other non-transitory form suitable for use in the implementation of processes according to embodiments. The carrier may be any entity or device capable of carrying the program, such as a RAM, a ROM, or an optical memory device, etc. The one or more processors of the apparatus 200 may comprise a central processing unit (CPU). The one or more processors may comprise a graphics processingunit (GPU). The one or more processors may comprise one or more of a fieldprogrammable gate array (FPGA), a programmable logic device (PLD), or a complex programmable logic device (CPLD). The one or more processors may comprise an application specific integrated circuit (ASIC). It will be appreciated by the skilled person that many other types of device, in addition to the examples provided, may be used to provide the one or more processors. The one or more processors may comprise multiple co-located processors or multiple disparately located processors. Operations performed by the one or more processors may be carried out by one or more of hardware, firmware, and software. The one or more processors may comprise data storage. The data storage may comprise one or both of volatile and non-volatile memory. The data storage may comprise one or more of random access memory (RAM), read-only memory (ROM), amagnetic or optical disk and disk drive, or a solid-state drive (SSD). It will beappreciated by the skilled person that many other types of memory, in addition to theexamples provided, may also be used. It will be appreciated by a person skilled in theart that the one or more processors may each comprise more, fewer and / or different components from those described. The techniques described herein may be implemented in software or hardware, or may be implemented using a combination of software and hardware. They may include configuring an apparatus to carry out and / or support any or all of techniques described herein. Although at least some aspects of the examples described herein with reference to the drawings comprise computer processes performed in processing systems or processors, examples described herein also extend to computer programs, for example computer programs on or in a carrier, adapted for putting the examples into practice. The carrier may be any entity or device capable of carrying the program. The carrier may comprise a computer readable storage media. Examples of tangible computer-readable storage media include, but are not limited to, an optical medium (e.g., CD-ROM, DVD-ROM or Blu-ray), flash memory card, floppy or hard disk or any other medium capable of storing computer-readable instructions such as firmware or microcode in at least one ROM or RAM or Programmable ROM (PROM) chips. Where in the foregoing description, integers or elements are mentioned which have known, obvious or foreseeable equivalents, then such equivalents are herein incorporated as if individually set forth. Reference should be made to the claims fordetermining the true scope of the present invention, which should be construed so asto encompass any such equivalents. It will also be appreciated by the reader that integers or features of the invention that are described as preferable, advantageous, convenient or the like are optional and do not limit the scope of the independent claims. Moreover, it is to be understood that such optional integers or features, whilst of possible benefit in some embodiments of the invention, may not be desirable, and may therefore be absent, in other embodiments.

Claims

Claims1. A computer-implemented method of generating a modified neural network anda trigger input, wherein the modified neural network can be identified using the trigger input, the method comprising: obtaining a trained neural network; determining an identification neuron for the trained neural network, theidentification neuron having one or more associated weights and biases and anassociated location in the modified neural network; implanting the identification neuron into the trained neural network in theassociated location to produce the modified neural network; andgenerating, using the modified neural network, the trigger input such that:when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; wherein the identification neuron is such that, when the identification neuronis triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network.

2. A method according to claim 1, wherein the determined identification neuronhas one or more randomly selected weights or biases.

3. A method according to claim 1 or 2, wherein generating the trigger inputcomprises: obtaining a base trigger input, the base trigger input being configured to at least partially trigger the identification neuron; and optimising the base trigger input to generate an optimised trigger input, wherein the identification neuron has a greater response to the optimised trigger input than to the base trigger input.

4. A method according to claim 3, wherein optimising the base trigger inputcomprises: iteratively generating one or more candidate trigger inputs;evaluating a response of the identification neuron to each of the one or more candidate trigger inputs; and on the basis of the one or more candidate trigger inputs and the evaluated responses, generating one or more further candidate trigger inputs that elicit an increased response from the identification neuron compared to the one or more candidate trigger inputs.

5. A method according to claim 4, wherein evaluating a response of theidentification neuron comprises evaluating an output of the modified neural network.

6. A method according to any preceding claim, wherein:the trained neural network is configured to, in response to being presented with a given input not comprising the trigger input, generate a corresponding given output; and the modified neural network is configured to, in response to being presented with the given input, generate the corresponding given output.

7. A method according to claim 6, wherein the modified neural network isconfigured to, in response to being presented with the given input modified toincorporate the trigger input, produce a different output to the corresponding givenoutput.

8. A method according to any preceding claim, wherein:the desired modification to the output of the modified neural network results ina modified output of the modified neural network; andthe method further comprises: recording the modified output; subsequent to the recording, presenting a further neural network with the trigger input, such that the further neural network generates a further output; and comparing the further output to the recorded modified output todetermine whether the further neural network is derived from the modifiedneural network.

9. A method according to any preceding claim, wherein an output of theimplanted identification neuron is connected to one or more further neurons in themodified neural network such that the triggering the implanted identification neuronactivates or deactivates a portion of the modified neural network.

10. A method according to claim 9, wherein the activation or deactivation of theportion of the modified neural network causes one or more functions of the modified neural network to be enabled or disabled.

11. A method according to any preceding claim, wherein triggering theidentification neuron comprises activating the identification neuron, such that: in the absence of the trigger input, the identification neuron remains relativelyinactive; and the identification neuron responds to the presence of the trigger input in aninput to the modified neural network by becoming relatively active.

12. A method according to any preceding claim 1 to 10, wherein the triggering theidentification neuron comprises deactivating the identification neuron, such that:in the absence of the trigger input, the identification neuron is relatively active; and the identification neuron responds to the presence of the trigger input in an input to the modified neural network by becoming relatively inactive.

13. A method according to any preceding claim, wherein the implanting comprisesadding an additional neuron to the trained neural network.

14. A method according to any of claims claim 1 to 12, wherein the implantingcomprises modifying one or more weights or biases of an existing neuron in the trained neural network.

15. A method according to any preceding claim, wherein the associated locationof the implanted neuron is in a hidden layer of the modified neural network.

16. A method according to any preceding claim, wherein the method comprisesdetermining the one or more associated weights and biases by repeating steps of: randomly selecting the one or more weights and biases, andevaluating whether the selected one or more weights and biases would cause the identification neuron to respond to a natural input to the trained neural network.

17. A method according to any preceding claim, wherein the method further comprises: obtaining operational input data; and embedding the trigger input into the obtained operational input data.

18. A method according to any preceding claim, wherein:the trained neural network is configured to perform one or more computer vision functions in respect of an input image; and the trigger input comprises at least a portion of the input image.

19. A method according to any preceding claim, wherein:the trained neural network is configured to generate text data in response toinput text prompts; and the trigger input comprises a text prompt.

20. A method according to any preceding claim, wherein:the method further comprises, subsequent to the implanting, performingfurther training of the modified neural network; andthe weights and biases of identification neuron persist through the furthertraining, such that, when the identification neuron is triggered by an input to the further trained neural network, it causes the desired modification to the output of the further trained neural network21. A method according to any preceding claim, wherein, when the identificationneuron is not triggered by an input to the modified neural network, the output of the modified neural network is not significantly modified.

22. A computer program product comprising instructions which, when theprogram is executed by a computing device, cause the computing device to perform a method according to any preceding claim.

23. A computer-implemented method of operating a modified neural network, theneural network having been modified by: determining an identification neuron for the trained neural network, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; and implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network, the method comprising: providing to the modified neural network, an input comprising a trigger input, the trigger input having been generated such that: when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; wherein the identification neuron is such that, when the identification neuronis triggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network, the desired modification being such that the modified neural network can be identified using the trigger input.

24. A computer program product comprising instructions which, when theprogram is executed by a computing device, cause the computing device to perform a method according to claim 23.

25. An apparatus comprising a modified neural network, wherein:the neural network has been modified by: determining an identification neuron for the trained neural network, the identification neuron having one or more associated weights and biases and an associated location in the modified neural network; and implanting the identification neuron into the trained neural network in the associated location to produce the modified neural network;the modified neural network is configured to receive a trigger input, the trigger input having been generated such that: when the trigger input is present in an input to the modified neural network, the identification neuron is triggered; and when the trigger input is not present in an input to the modified neural network, the identification neuron is not triggered; and the identification neuron is such that, when the identification neuron istriggered by an input to the modified neural network, it causes a desired modification to the output of the modified neural network, the desired modification being such that the modified neural network can be identified using the trigger input.

Images