Method and system for image analysis and recommendation

Abstract

Method of scoring images, comprising: analysing an image(s) using a convolutional neural network to identify visual features 802; scoring images based on the visual features via a profile recommendati

Description

TECHNICAL FIELD This invention relates to a method and system for image analysis, scoring, ranking and recommendation using a Convolutional Neural Network (CNN). BACKGROUND When faced with multiple images, human beings can often struggle to consciously identify factors that may make one image more appealing over another. In some cases they may not even be aware of these factors in general. This can be problematic when the user is required to select a single image or a subset of images from the multiple images to best represent themselves, for instance for use as a "profile picture" on a dating application or social media profile. In these examples, certain visual factors of an image have been shown to be more appealing than others. Accordingly, there is a need for a system that can identify these factors and assist users when faced with this problem. SUMMARY OF THE INVENTION A first aspect of the invention provides a method of scoring one or more images comprising: providing one or more images to a convolutional neural network; analysing the one or more images using the convolutional neural network to identify one or more visual features in each of the one or more images; assigning, via a profile recommendation policy, a score to each of the one or more images based on the identified one or more visual features; receiving feedback on the one or more scores; and updating the profile recommendation policy of the convolutional neural network based on the feedback. In some embodiments, the method (and any of the other methods described and claimed herein) may be computer-implemented. Although it has been described that in some embodiments a convolutional neural network is used, it will be appreciated that in other embodiments any neural network or system suitable for extracting visual features from an image could alternatively be used. Mention of a convolutional neural network throughout this document should be understood with that in mind. Throughout this document references to methods and systems may be interpreted as meaning computer-implemented methods and computer-implemented systems. Additionally or alternatively, references to methods and systems may be interpreted as meaning computer-implemented machine learning methods and computer-implemented machine learning systems. The method may further comprise ranking, via the profile recommendation policy, the one or more images based on the scores assigned to the one or more images. The method may further comprise recommending, via the profile recommendation policy, a first image to a user based on the one or more scores assigned to the one or more images. This recommendation may be provided to the user by way of a display, user interface, GUI, or any combination of these. The user feedback may be received via the display, user interface or GUI. Recommending the first image to the user may comprise recommending that the user uses the first image as a profile picture for a social media profile, dating application profile or website profile. The first image may be the image with the highest score from the one or more images. The one or more images may comprise 2-10 images inclusive. In other words, there may be 2 or 10 images, or any number therebetween. Each of the one or more images may be a photograph of a human being. Each of the one or more images may specifically be a photograph of the user. The one or more visual features may be indicative of one or more of the following features of the human being: facial expression; attractiveness; age; gender; and clothing style. The one or more visual features may be indicative of one or more of the following technical features of the image: resolution; lighting conditions; and blurriness. The one or more visual features may be indicative of one or more of the following aesthetic features of the image: image feature distribution; image feature composition; use of the rule of thirds; colour distribution; colour harmony; feature balance; and framing. The method may further comprise pre-processing image data to produce the one or more images. Pre-processing the image data may comprise performing one or more of the following operations on the image data in order to produce the one or more images: image resizing; image normalisation; data augmentation; and image transformation. The step of analysing may comprise the convolutional neural network extracting visual features from the one or more images into a representation that can be processed by a reinforcement learning model. The reinforcement learning model may be adapted to implement the profile recommendation policy. The step of processing the visual features using a reinforcement learning model may comprise inputting the one or more visual features into a neural network adapted for reinforcement learning, such as a multilayer perceptron. The neural network may comprise a plurality of layers, such as 3 or 4 layers. The feedback may comprise a user selection of an image of the plurality of images. The feedback may comprise a user selection of the same image as the recommended image. The feedback may comprise a user selection of a different image to the recommended image. The feedback may be received by a reward model. The reward model may provide a reward based on the feedback. The profile recommendation policy may be updated based on the reward. The reward may comprise one of a positive reward and a negative reward. The reward may comprise a reward function. The reward model may learn the reward function from user feedback. The step of said reward model learning said reward function may comprise the use of inverse reinforcement learning. The method may further comprise the step of training said reward model according to supervised learning using feedback on image scores provided by a plurality of users. The reward model may comprise a machine learning model trained to predict a score for each image. The reward model may provide said reward to a Proximal Policy Optimization (PPO) algorithm. The PPO may update the profile recommendation policy based on the reward. The one or more visual features may comprise a plurality of visual features, and wherein updating the profile recommendation policy comprises adjusting relative weights assigned to each of the plurality of visual features. The method may further comprise iterating the method at least 1,000 times, at least 10,000 times, at least 100,000 times, or more. Each of these iterations may be carried out with different images and different users. The method of the first aspect may be a computer-implemented method. An additional aspect of the invention provides a method of ranking a plurality of images comprising: providing a plurality of images to a convolutional neural network; analysing the plurality of images using the convolutional neural network to identify one or more visual features in each of the images; assigning, via a profile recommendation policy, a rank to each of the images based on the identified one or more visual features; receiving feedback on the ranks; and updating the profile recommendation policy of the convolutional neural network based on the feedback. In some embodiments, the method may be computer-implemented. A second aspect of the invention provides a system for scoring images comprising: a convolutional neural network configured to: receive one or more images; analyse the one or more images to identify one or more visual features in each of the one or more images; a profile recommendation policy configured to: assign a score to each of the one or more images based on the identified one or more visual features; and an updating algorithm configured to: receive feedback on the one or more scores; and update the profile recommendation policy based on the feedback. In some embodiments, the system may comprise at least one processor and a memory storing computer-executable instructions. The profile recommendation policy may be further configured to rank the one or more images based on the scores assigned to the one or more images. The profile recommendation policy may be further configured to recommend a first image to a user based on the one or more scores assigned to the one or more images. The system may further comprise a display, user interface, GUI, or any combination of these, to provide this recommendation to the user. The user feedback may be received via the display, user interface or GUI. The feedback may be provided by a user. The feedback may be an indication of the user's preferred image. The system may further comprise a reinforcement learning model adapted to implement the profile recommendation policy. The reinforcement learning model may comprise a neural network adapted for reinforcement learning, such as a multilayer perceptron. The neural network may comprise a plurality of layers, such as 3 or 4 layers. The system may further comprise a reward model configured to receive the feedback on the one or more scores, and generate a reward based on the feedback. The updating algorithm may be configured to update the profile recommendation policy based on the reward. The updating algorithm may be a Proximal Policy Optimization (PPO) algorithm. The system may include a CNN head. The CNN head may be configured to perform one or more convolution operations. The system may include a dropout layer. The dropout layer may be configured to randomly set a fraction of input units to 0 during training time. The system may include one or more densely connected layers The system may include one or more Rectified Linear Unit (ReLU) activation functions. The ReLU activation function(s) may be configured to introduce non-linearity into the system model / algorithm. The system may include a sigmoid activation function. The sigmoid activation function may be configured to constrain each score to a number in the range [0,1]. A third aspect of the invention provides a method of training a system comprising: receiving one or more images at a convolutional neural network; identifying, using the convolutional neural network, one or more features of each of the one or more images; assigning, using a profile recommendation policy, a weighting to each of the one or more features; receiving, at a PPO, feedback on each of the one or more images; iteratively adjusting the weighting of each feature based on the received feedback. The method may be repeated a number of times. The method may be repeated 1000, 10000 or 100000 times. The iterative adjustment of the weightings may be made so as to improve further feedback if the method were repeated. A fourth aspect of the invention provides method of training a reward model to predict image scores comprising: receiving, at the reward model, a plurality of images; receiving, at the reward model, human feedback comprising an actual score corresponding to each of the plurality of images; generating, by the reward model, a predicted score for each of the plurality of images; training the reward model based on the plurality of images, the predicted scores and the actual scores, so as to minimise a difference between each predicted score and each actual score for a given image of the plurality of images. A fifth aspect of the invention provides a computer-readable medium containing instructions which, when read by a computer, cause the computer to perform the method of the first aspect using a convolutional neural network trained by the method of the third aspect. The method may also or alternatively use a reward model trained by the method of the fourth aspect. The computer-readable medium may be a non-transitory computer-readable medium. A sixth aspect of the invention provides a server (including, for example, a cloud server) for processing a plurality of different digital images of the same user, which server comprises a processor and a memory storing computer-executable instructions that, when executed by the processor, cause the server to perform the steps of: receiving a plurality of different digital images of the same user; 5 transmitting said plurality of different digital images to a system according to the second aspect set out above, or any system according to the invention described anywhere herein; receiving from said system at least one score, the or each score associated with a corresponding one of said plurality of different digital images; providing to a user of said system an output representing said at least one score; receiving from said user an indication of a selected one digital image of said plurality of different digital images; storing in said memory an association between said selected one digital image and a user profile of said user; and providing an indication to said system of said selected one digital image. In some embodiments the association facilitates retrieval and transmission of the selected one digital image to the user and to other users when the user profile is accessed. In some embodiments, the second system may be remote from said server. For example, the server may provide access to a website, and / or provide data for a remote application running on user device, the website and / or remote application permitting a user to view their own user profile and / or user profiles of other users. In some embodiments the server may comprise the second system. Some embodiments of the invention relate to the field of computer-implemented methods and computer-implemented systems. Some embodiments of the invention relate to the field of computer-implemented methods and computer-implemented systems for digital image analysis, such as digital image scoring and / or ranking and / or classification and / or categorisation. Some embodiments of the invention relate to the field of computer-implemented methods and computer-implemented systems for digital image analysis for use in creating and modifying online user profiles, for example on social media or any platform, website or application (sometimes known as an 'app') that permits a user to choose a picture that is used in their online user profile. Typically such apps are available to the user from online stores including the Apple 'App Store' and Google Play. Some embodiments of the invention relate to the field of computer-implemented methods and computer-implemented systems for digital image analysis in the field of dating websites and applications. According to some embodiments, there is provided a computer-implemented system for scoring or ranking a digital image of a user, which computer-implemented system comprises: a convolutional neural network trained to extract features from said digital image (or any other neural network adapted to perform this function), and to output said features; a reinforcement learning algorithm or model adapted to receive said features output from the convolutional neural network, the reinforcement learning algorithm comprising a profile recommendation policy for scoring or ranking said digital image, the reinforcement learning algorithm further adapted to output said score or rank to a user; a reward model comprising a reward function, the reward model adapted to: receive or generate feedback indicating a digital image selected by the user; update said reward function based on said feedback whereby said reward model learns said reward function according to said feedback; and output rewards according to the reward function; and a Proximal Policy Optimisation algorithm adapted to receive the rewards from the reward model and to adjust the profile recommendation policy of the reinforcement learning algorithm according to the rewards. Throughout the description and claims of this specification, the words "comprise" and "contain" and variations of the words, for example "comprising" and "comprises", mean "including but not limited to", and do not exclude other components, integers or steps. Moreover the singular encompasses the plural unless the context otherwise requires: in particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise. Preferred features of each aspect of the invention may be as described in connection with any of the other aspects. Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 is a flow diagram showing a method according to an embodiment of the invention; Figure 2 shows a pre-processing step of data augmentation; Figure 3 is a flow diagram showing a more extended method according to the embodiment of Figure 1; Figure 4 shows a first pre-processing step; Figure 5 shows a second pre-processing step; Figure 6 shows the operation of a convolutional neural network; Figure 7 shows the result of a method according to an embodiment of the invention; Figure 8 shows a system according one embodiment of the invention; and Figure 9 shows a system implementation according to one embodiment of the invention. DETAILED DESCRIPTION This description is divided roughly into two sections: a first section describing the use of various neural networks that have been trained, and a second section explaining the training of those neural networks before being used on new data. Figure 1 shows an algorithm 100 for recommending an image to a user performed by a convolutional neural network (CNN). The recommended image may be used as a "profile picture" or a main / primary picture for a dating application profile or a social media profile, for example. Hence, the images are likely photographs of the user, such as a photograph of the user's face, head and / or body. The CNN may be hosted on a cloud, for example Amazon Web Services (AWS) or Google Cloud Platform (GCP). A suitable framework may be created using PyTorch, for example. In general, an "image" refers to a matrix of pixels, each pixel typically having a red, green and blue component. In this invention, digital images are used. The digital images may be provided in any digital format, such as a "jpeg" file or a "png" file for example. The CNN receives one or more images 102 from which one or more of the images is to be selected by the algorithm and recommended to the user. In practice the number of images received will likely be in the region of 2-10 inclusive, although only one image may be provided, or more than 10 images may be provided. The images may have been pre-processed, which is discussed in further detail below. Once the images have been received, the CNN analyses the images in order to identify one or more visual features in each of the images 104. The visual features being identified are indicative of aspects of the image and / or subject of the image (often the user themselves) that influence how each image scores, and therefore whether it is suitable to be recommended to the user. Specifically, the visual features are indicative of features of the subject, technical features of the image, and aesthetic features of the image. Features of the subject that can be determined by the visual features include facial / emotional expressions (such as smiling, happiness, sadness, surprise or laughing, for example), attractiveness (which may be a perceived attractiveness or alternatively an attractiveness determined by the so-called "golden ratio", for example), age, gender, and clothing style. Technical features of the image that can be determined by the visual features include image resolution, lighting conditions and shadows, and blurriness. Aesthetic features of the image that can be determined by the visual features include aesthetically pleasing image feature distribution and composition such as the use of the "rule of thirds", colour distribution, colour harmony, feature balance, and framing. Following identification of the visual features, a profile recommendation policy assigns a score to each image based on a weighting applied to each of the visual features. The relative weightings of the visual features are used to show that some features are considered more important than others in terms of making up a high scoring image, and hence one that may be recommended to the user. The profile recommendation policy is a Reinforcement Learning, RL, algorithm with a policy. It is a neural network, such as a multilayer perceptron having 3-4 intermediate layers. These intermediate layers are dense layers, which are discussed in more detail below. This neural network is trained using user feedback on multiple sets of images. Based on the identified visual features and their respective weightings, the profile recommendation policy assigns each image a score 106. The score is typically normalised so that it is a number falling between 0 and 1, with 1 being a higher / better score than 0. Any technique of differentiating the images from one another can be used as a 'score'. The scores of the one or more images are compared and the highest scored image is identified and recommended to the user 108. In some embodiments, multiple images may be recommended to the user, these being the highest scoring images. Turning now to Figure 3, a more extended method 200 is shown which uses the principles described with respect to Figure 1. The CNN used in Figure 3 has been trained in the same way as the CNN used in Figure 1. Image data 202 is provided to a pre-processing system. The image data comprises a number of raw, unprocessed images to be scored. The preprocessing system pre-processes the image data 204 to prepare the data for feeding into the CNN. The pre-processing includes resizing the images to a consistent size to ensure uniformity in the input data. This step is shown in Figure 4. Six input images are provided to the pre processing unit. They are resized to a consistent size. Typically, the images are resized to a specific resolution, 224x224 pixels, for example. The pre-processing further comprises normalising the images. This step involves scaling the pixel values of the images. Typically, the pixel values are scaled to a range of 0 to 1. This step is shown in Figure 5, in which the raw image data is normalised so that each pixel value comprises a number between 0 and 1. An alternative scaling may result in the pixel values having a zero mean and unit variance. It is noted that the pre-processing steps described above may be performed by another system or and the one or more images may be received in Figure 3 in a format ready to use in step 206 without any further processing. Following any image pre-processing that has taken place, the resulting one or more images are provided to the CNN for image analysis and visual feature identification 206. Figure 6 shows the manner in which a single image may be typically processed by the CNN. An image is received as an input. This image has red, green and blue components and hence there are three colour planes 602 defining each of the three colour matrices input into the CNN. A first convolution is performed on the colour planes 602, using a 5x5 size kernel. For three 32x32 colour planes, this results in 32 18x18 matrices 604. A first max pooling operation is then performed on these matrices, using a 2x2 size kernel. The max pooling operation uses the maximum value from the kernel at each location in the matrix to generate the resulting matrix. Hence, this operation results in 32 10x10 matrices 606. A further convolution operation is performed using a 5x5 kernel, resulting in 48 6x6 matrices 608. A further max pooling operation is performed using a 2x2 kernel, resulting in 48 4x4 matrices 610. These matrices are flattened to give 768 units 612. These units are then passed through two fully connected layers to ultimately provide an output 614. The output 614 is a visual feature that has been identified in the image. The output may alternatively comprise multiple visual features that have been identified in the image. Turning to the architecture of the CNN, this comprises the following components: a CNN head; a dropout layer; densely connected layers (multilayer perceptron); a ReLU activation function; and a sigmoid activation function. These layers typically involve a series of convolutional, pooling and normalisation operations. The CNN head acts as the primary feature extractor in the network. The equation for a convolutional operation performed by the CNN head is as follows: F(i,j) = G(m,n)H(i - m,j — n) m n where F(I,j) is the pixel in the output feature map, G(m,n) is the input image, and H(i-m,j-n) is the kernel or filter being convolved with the input image. The dropout layer is included to reduce the likelihood of overfitting, by randomly setting a fraction of the input units to 0 during training time. The equation for a dropout operation is: y = D (x, y) = xB (1 — y) where x is the input to the dropout layer, y is the fraction of the input units to drop, and B(l-y) is a Bernoulli distribution which returns 1 with probability (1-y). Densely connected layers are responsible for further processing the features extracted by the CNN head. Dense layers are fully connected, and therefore receive an output from every neuron of the preceding layer. They use matrix-vector multiplication to precisely transform the dimensionality of the output of the previous layer to allow the model to define the relationship the values of the data in which the model is working. In other words, the output of a dense layer is a chosen N-dimensional vector. Hence, dense layers can be used to reduce the dimensions of the vectors, and use every neuron from the preceding vector in doing so. The output from the dropout layer is flattened and then passed through one or more fully connected layers, as discussed above. The equation for densely connected layer operation is as follows: y = u(Wx + b) where W is the weight matrix, x is the input, b is the bias vector, and cr is the activation function. Between each of these intermediate layers, a Rectified Linear Unit (ReLU) function is used. This function introduces non-linearity into the model, which enables it to learn more complex patterns. The equation for a ReLU operation is: y = max (x, 0) where x is the input to the function. In the final layer, a sigmoid activation function is used to constrain the output to a range between [0, 1], representing the probability distribution of the image being the best choice. The equation fora sigmoid operation is: y = 1 / (1 + e-) where x is the input to the function. Using these layers, the CNN is instantiated in the following order: 1. CNN Head 2. ReLU activation 3. Dense layer 4. ReLU activation 5. Dropout layer 6. Dense layer 7. ReLU activation 8. Dense layer 9. Sigmoid activation The sizes of each layer could be determined using a Bayesian optimization algorithm during production. Once the images have been analysed and the visual features identified, the profile recommendation policy scores each of the images 208, based on the identified visual features. The result of this is shown in Figure 7, in which the aforementioned visual features have been used to assign each of the images a score. The score has been normalised so that it falls between 0 and 1. The images are then ranked according to these scores. In the example of Figure 7 therefore, the image scored 0.97 would be ranked first and the image scored 0.32 would be ranked sixth. Based on this ranking, an image is recommended to the user by the profile recommendation policy. Again, referring to the example of Figure 7, the image scored 0.97 would be recommended to the user. Returning now to Figure 3, following the scoring and recommendation phase 208, the user sees the image recommendation and chooses their preferred image 210. This choice may be to make that image their chosen profile picture for a dating application profile, a dating website profile or a social media profile, for example. The user may select the image that was recommended by the profile recommendation policy, or they may select a different image. They may select multiple images. The user's selection(s) are used as feedback in a Reinforcement Learning with Human Feedback (RLHF) step 212. This step implements a reward model and a Proximal Policy Optimization (PPO) algorithm to update, develop and train the profile recommendation policy. In a first scenario, if the user selects the image that was recommended by the profile recommendation policy, the PPO receive a positive reward from the reward model. The positive reward causes the PPO to reinforce the profile recommendation policy and weightings that were applied to each of the visual features. This reinforcement may involve not altering the profile recommendation policy at all. Alternatively, the reinforcement may involve altering the profile recommendation policy to provide more weight to the features in the selected image. In a second scenario, if the user selects an image that was not the image (or one of the images) recommended by the profile recommendation policy, the PPO receives a negative reward from the reward model. The negative reward alters the weightings of the visual features, such that for example, the features that were deemed important in the recommended image(s) (i.e. features with a high weighting) are now considered less important (their weightings are reduced), and the weightings of the features that were present in the user-selected image are increased. In a third scenario, if the user selects multiple images, a more complex reward function may be provided to the PPO by the reward model such that the weightings associated with the visual features contained in the selected images are increased and the weightings associated with the visual features contained in the unselected images are decreased. In this manner the profile recommendation policy is updated as users use the algorithm to choose an image fortheir user profile. Following this RLHF step, the updated profile recommendation policy may be used to analyse and score a new set of images. The entire process, including the RLHF, may be repeated many times with many different sets of images, with many different users. It may be repeated thousands of times, tens of thousands of times, hundreds of thousands of times, or millions of times, for example. As discussed above, the updating is performed by the PPO. This is an algorithm that uses a surrogate function to achieve the optimization of a policy in RL. PPO has the advantage of achieving stable and reliable performance while using fewer resources compared to other RL algorithms. PPO can be used to quantitatively improve the profile recommendation policy based on the rankings of the images. The primary equation of PPO is the clipped objective function / (0). If the policy is denoted by n and the advantage function by A (which measures how much better an action is compared to the average action of that state), then the original objective of policy gradient methods is to maximise / (0) = Et[ where at is the action, st is the state, n is the parameterized policy, At is the advantage function. In this case, the actions are the normalised ranking of the image, the state is the current embedded image being analysed, and the policy networks are composed by the CNN architecture, discussed above. However, in PPO, the objective function / (0) is modified to limit the policy update: / / ?r(aJs,) \ ( 7r(aJs,;0) \ \ / (0) = Et min ...................................r........J,clip k....................................r.........;.....z.........................r, 1 - f.l - f T The clip function ensures that the ratio between the new policy and the old policy, —js close to 1, which means the new policy should not deviate largely from the Koid(at\st;0oidY old policy, e is a hyperpara meter that controls the range of the clip function. In this application, this ensures that the recommendation algorithm is improved in a monotonic and stable manner as the user dataset grows. The PPO usually uses two types of networks: policy network and value network. The policy network is responsible for generating actions given the state, and the value network is responsible for estimating the value of a state. Both networks are updated using the PPO algorithm. In this case, the CNN can be seen as a feature extractor that transforms raw images into a representation that can be processed by the policy and value networks. The reward model provides feedback (rewards) based on the scoring and ranking of the images. PPO then uses this feedback to update the policy and value networks so that better-ranked images are more likely to be selected in the future. Figure 8 shows an embodiment of the system that has been described above. The system is shown generally at 800. One or more images to be scored are received at a CNN 802. The CNN 802 analyses the one or more images and identifies one or more visual features in each of the one or more images. Based on these identified visual features, a profile recommendation policy 804, which is a reinforcement learning (RL) system comprising a policy, scores and ranks the images, and accordingly recommends an image (or multiple images) to a user. This recommendation is provided to the user by way of a user interface 806, which displays the selection of images for the user to choose from. The user device on which the user interface is shown is not considered to be a part of system 800. The user provides feedback on the recommendation by way of the user interface 806. Note that the feedback does not have to be 'active' in the sense that the user knows feedback is being given. The feedback can be inferred from a user behaviour observable by the system. For example, the observable behaviour could be the image the user actually selects from the set of images input into the system. The behaviour is observed by a reward model 808. The reward model 808 uses this observed behaviour to updated the reward function. The updated reward function is transmitted to a PPO 810. The PPO 810 receives the updated reward function and updates the profile recommendation policy 804 based on the reward function. In this way the reward function is updated over time as the feedback from each user (when an image is chosen) is used to update the reward function, and thereby adapt the policy applied by the RL to score each image. This approach has the advantage of being able to apply RL in complex environments where it is difficult to manually specify a suitable reward function. In other words, the reward model is able to provide a suitably complex reward function to the PPO, even when faced with a relatively simple observed behaviour from the user. Figure 9 shows an embodiment of the system in use. A user will communicate with the system server 904 with their user device 902a. This user device 902a could be a mobile phone, a desktop computer, a tablet or a laptop for example. The user provides a number of images to the server 904, which in turn provides these images to the system 800, in order to analyse, score and / or rank these images, and recommend an image, in the manner discussed above. The recommended image, and optionally the image scores / ranks, are returned to the server 904. The recommended image, and optionally the image scores / ranks are then returned to the user via their user device 902a. The server 904 will be in communication with a number of users via their respective user devices 902a, 902b, 902c, and will be able to receive images from each of these users in the manner described above. Training - CNN Prior to using the algorithm (e.g. as shown in Figures 1 and 3) on new images (i.e. images not previously seen by the CNN), it is necessary to train the CNN. To do that, a number of different training images are collected. Typically at least 500 training images are collected. These training images may be photographs of real human users and / or they may be computer-generated images of humans, generated by a generative model such as "Midjourney", for example. Optionally, these training images undergo a pre-processing step of data augmentation. As shown in Figure 2, new augmented images are generated by performing random transformations on the existing images (i.e. on the original images or on the already-generated augmented images). These random transformations may include rotations, translations, reflections (i.e. flips), stretch operations and scaling / zooming operations. This pre-processing step may be used during the initial training phase of the CNN to increase the diversity and robustness of the data being fed into the CNN, meaning that the CNN will be exposed to a wider range of variations. The pre-processing may also comprising resizing and / or normalising the images as described above. Using the pre-processed training images, the CNN is trained to recognise and interpret important elements in the images, such as emotional expressions, image quality, aesthetics and composition. The training process uses backpropagation and optimization algorithms, such as an adaptive moment estimation algorithm. Through the training, the CNN iteratively adjusts its weights to better predict the desired characteristics. Training - reward model Prior to use, the reward model 808 is trained using training data 812. The training data 812 comprises different training images, each training image associated with human feedback, i.e. a manual score on the basis of the relevant visual features, such as technical and / or aesthetic features. The different training images may be photographs of users or they may be computergenerated images of human beings, generated by a generative model such as "Midjourney", for example. Typically, at least 500 images will be collected. The reward model is defined as R(s,a), which takes a state-action pair (s,a) as an input and outputs a predicted reward. Here, s represents the state (i.e. the embedding of the image presented) and a represents the action (i.e. the selected image). R(s,a) = r, where r e R is the predicted reward. The dataset of human feedback on different training images include state-action-reward tuples (s,a,r), where r is the human-provided score for action a in state s. k supervised learning algorithm is used to train the reward model. The objective is to minimise the difference between the predicted reward R(s,a) and the actual reward r. This can be formalised as the following loss function: L = E[(r(s,a) — r)2], where E represents the expectation over the dataset of human feedback. Specifically, the reward model is derived using the PPO algorithm. The PPO algorithm trains a parameterised neural network (i.e. a neural network that can be defined or represented in terms of one or more parameters) that can have as inputs interactions from the user, and outputs a reward indicating the likelihood of future interactions with the given image(s). Once the reward model is trained, it can be used to provide rewards for the PPO algorithm. As the PPO algorithm improves, it may be necessary to update the reward model accordingly. This can be done by periodically collecting more human feedback on new training images and retraining the reward model.

Claims

1. A method of scoring one or more images comprising:providing one or more images to a convolutional neural network;analysing the one or more images using the convolutional neural network to identify one or more visual features in each of the one or more images;assigning, via a profile recommendation policy, a score to each of the one or more images based on the identified one or more visual features;receiving feedback on the one or more scores; andupdating the profile recommendation policy based on the feedback.

2. The method of claim 1, further comprising ranking, via the profile recommendation policy, the one or more images based on the scores assigned to the one or more images.

3. The method of claim 1 or 2, further comprising recommending, via the profile recommendation policy, a first image to a user based on the one or more scores assigned to the one or more images.

4. The method of claim 3, wherein recommending the first image to the user comprises recommending that the user uses the first image as a profile picture for a social media profile, dating application profile or website profile.

5. The method of any of claims 2-4, wherein the first image is the image with the highest score from the one or more images.

6. The method of any preceding claim, wherein the one or more images comprises 2-10 images inclusive.

7. The method of any preceding claim, wherein each of the one or more images is a photograph of a human being.

8. The method of claim 7, wherein the one or more visual features are indicative of one or more of the following features of the human being: facial expression; attractiveness;age; gender; and clothing style.

9. The method of any preceding claim, wherein the one or more visual features are indicative of one or more of the following technical features of the image: resolution; lighting conditions; and blurriness.

10. The method of any preceding claim, wherein the one or more visual features are indicative of one or more of the following aesthetic features of the image: image feature distribution; image feature composition; use of the rule of thirds; colour distribution; colour harmony; feature balance; and framing.

11. The method of any preceding claim, further comprising pre-processing image data to produce the one or more images.

12. The method of claim 11, wherein pre-processing the image data comprises performing one or more of the following operations on the image data in order to produce the one or more images: image resizing; image normalisation; data augmentation; and image transformation.

13. The method of any preceding claim, wherein said step of analysing comprises said convolutional neural network extracting features from said one or more images into a representation that can be processed by a reinforcement learning model.

14. The method of any preceding claim, wherein the feedback comprises a user selection of an image of the plurality of images.

15. The method of any preceding claim, wherein:the feedback is received by a reward model;the reward model provides a reward based on the feedback; andthe profile recommendation policy is updated based on the reward, optionally where said reward comprises one of a positive reward and a negative reward.

16. The method of claim 15, wherein the reward model comprises a reward function, the method further comprising the step of said reward model learning said reward function from user feedback.

17. The method of claim 16, wherein said step of said reward model learning said reward function comprises the use of inverse reinforcement learning, and optionally comprises the step of using a neural network adapted to utilise reinforcement learning such as a multilayer perceptron.

18. The method of claim 15, 16 or 17, further comprising the step of training said reward model according to supervised learning using feedback comprising manually scored images.

19. The method of any of claims 15 to 18, wherein said reward model comprises a machine learning model trained to predict a score for each image.

20. The method of any of claims 15 to 19, wherein the reward model provides said reward to a Proximal Policy Optimization (PPO) algorithm, and wherein the PPO updates the profile recommendation policy based on the reward.

21. The method of any preceding claim, wherein the one or more visual features comprises a plurality of visual features, and wherein updating the profile recommendation policy comprises adjusting relative weights assigned to each of the plurality of visual features.

22. The method of any preceding claim, further comprising iterating the method at least 1,000 times.

23. The method of claim 18, comprising iterating the method at least 10,000 times.

24. The method of claim 19, comprising iterating the method at least 100,000 times.

25. A system for scoring images comprising:a convolutional neural network configured to:receive one or more images;analyse the one or more images to identify one or more visual features in each of the one or more images;a profile recommendation policy configured to:assign a score to each of the one or more images based on the identified one or more visual features;an updating algorithm configured to:receive feedback on the one or more scores; andupdate the profile recommendation policy based on the feedback.

26. The system of claim 25, wherein the profile recommendation policy is further configured to rank the one or more images based on the scores assigned to the one or more images.

27. The system of claim 25 or 26, wherein the profile recommendation policy is further configured to recommend a first image to a user based on the one or more scores assigned to the one or more images.

28. The system of any of claims 25 to 27, wherein the feedback is provided by a user via a user interface.

29. The system of any of claims 25 to 28, wherein said updating algorithm comprises a reinforcement learning algorithm adapted to implement said profile recommendation policy, and optionally wherein said reinforcement learning algorithm comprises a neural network, such as a multilayer perceptron comprising a plurality of layers, for example 3 or 4 layers.

30. The system of any of claims 25 to 29, further comprising a reward model configured to receive the feedback on the one or more scores, and generate a reward based on the feedback, wherein the updating algorithm is configured to update the profile recommendation policy based on the reward.

31. The system of any of claims 25 to 30, wherein the updating algorithm comprises a Proximal Policy Optimization (PPO) algorithm.

32. A method of training a system comprising:receiving one or more images at a convolutional neural network;identifying, using the convolutional neural network, one or more features of each of the one or more images;assigning, using a profile recommendation policy, a weighting to each of the one or more features;receiving, at a PPO, feedback on each of the one or more images;iteratively adjusting the weighting of each feature based on the received feedback.

33. A method of training a reward model to predict image scores comprising:receiving, at the reward model, a plurality of images;receiving, at the reward model, human feedback comprising an actual score corresponding to each of the plurality of images;generating, by the reward model, a predicted score for each of the plurality of images;training the reward model based on the plurality of images, the predicted scores and the actual scores, so as to minimise a difference between each predicted score and each actual score for a given image of the plurality of images.

34. A computer-implemented method of updating a profile recommendation policy for scoring images using a reinforcement learning algorithm, which method comprises the steps of:receiving, at a reward model optionally trained according to the method of claim 33, feedback indicating an image selected by a user;using said feedback to update a reward function of said reward model to generate an updated reward function; andupdating said profile recommendation policy according to said updated reward function.

35. A computer-implemented method according to claim 34, further comprising the steps of receiving further feedback at said reward model, using said further feedback to further update said reward function to generate a further updated reward function, and updating said profile recommendation policy according to said further updated reward function.

36. A computer-readable medium comprising instructions which, when read by a computer, cause the computer to perform the method of any of claims 1-24.

37. A computer-readable medium comprising instructions which, when read by a computer, cause the computer to perform the method of claim 32.

38. A computer-readable medium comprising instructions which, when read by a computer, cause the computer to perform the method of claim 33.

39. A computer-readable medium comprising instructions which, when read by a computer, cause the computer to perform the method of claim 34.

Images