System and method for the detection and identification of a body using multiviews models

Abstract

A computer-implemented method for identifying an object or a pose thereof using a machine learning algorithm based upon a neural network architecture comprises providing input data S101 representing 2

Description

TECHNICAL FIELD The present invention relates to a computer implemented method for identifying a body or a pose of a body using a machine learning algorithm based upon a neural network architecture. The present invention furthermore relates to a computer program and a computer readable medium comprising a computer program for carry out said method. Also, the present invention relates to system for identifying a body or a pose of a body. BACKGROUND Object identification using images is a common task in computer vision and machine learning. There are several methods and algorithms available for this purpose. A common approach to identify objects using images comprises a data collection for gathering a dataset of images that contains the objects to be identified. The dataset could include images with labeled annotations indicating the presence and location of the objects of interest. The images are then processed for enhancing the quality and reducing noise. This may involve resizing, normalizing, and applying filters to the images. Additionally, background removal or feature extraction can be performed depending on the specific requirements of the task. The relevant features can be extracted from the preprocessed images. Features can be simple attributes like color, texture, or shape, or more complex features extracted using deep learning techniques such as convolutional neural networks (CNNs). The dataset can then be split into training and validation sets. The training set is used to train a machine learning model or a deep learning model. The choice of model depends on the complexity of the task and the size of the dataset. Popular models for object identification include CNN architectures like VGG, ResNet, or MobileNet. In order to improve the performance, the model parameters can be fine-tuned. This may involve adjusting hyperparameters, optimizing the training algorithm, or using techniques like data augmentation to increase the diversity of the training data. The trained model is validated using the validation set to measure its performance. Common evaluation metrics include accuracy, precision, recall, and F1 -score. Once the model achieves satisfactory performance, it can be used to identify objects in new, unseen images. Test images are fed into the trained model and the model's predictions are used to identify the objects of interest. While the method for identifying objects using images is widely used and effective, it does have some drawbacks. As mentioned above, deep learning models, such as CNNs, are commonly used for object identification. These models often have millions of parameters and require significant computational resources for inference. The complexity of these models can result in slower inference speeds, especially when deploying them on resource-constrained devices or in real-time applications. Also, object identification using images often requires significant computational resources. Training complex models or processing large amounts of data can be computationally intensive, requiring specialized hardware or cloud computing resources. Furthermore, present object identification models can be sensitive to variations in image quality, lighting conditions, and viewpoint. As a matter of fact, an accurate object identification can be strongly affected by images taken from different angles. Examples of the present disclosure seek to address or at least alleviate the above problems. SUMMARY In a first aspect, there is provided a computer implemented method for identifying a body or a pose of a body using a machine learning algorithm based upon a neural network architecture, the method comprising: providing input data representing at least one 2D image of the body taken from a view point; applying a machine learning model to the input data; and obtaining a digital representation of the body or of the pose of the body as output data, wherein the machine learning model includes a neural network architecture that incorporates a two-head architecture Multiview model treating the view point of the image and a category of the body, or a category of the pose, as two independent variables, for extracting features of the body from different view points. In a second aspect, there is provided a computer program, comprising instructions which, when the program is executed by a computing device, cause the computing device to carry out the computer-implemented method according to the first aspect. In a third aspect, there is provided a computer readable medium comprising a computer program for carrying out the method according to the first aspect. In a fourth aspect, there is provided a system for identifying a body or a pose of a body using a machine learning algorithm based upon a neural network architecture using the method according to the first aspect, the system comprising a plurality of 2D cameras for monitoring the body and taking at least two 2D images by one camera at fixed time interval or taking a plurality of 2D images by a plurality of cameras, positioned at fixed distances. Other aspects and features are defined in the appended claims. Examples of the disclosure may provide a method and a system to detect and identify a body, or a pose of a body, in an efficient way irrespective of different viewpoints. Also, examples of the disclosure may provide a method and a system for identifying a body using a machine learning model with high degree of scalability and reduced inference speeds. In particular, examples of the disclosure may provide a method and a system for identifying a body or a pose of a body with an automated and faster learning process. BRIEF DESCRIPTION OF DRAWINGS Examples of the disclosure will now be described by way of example only with reference to the accompanying drawings, in which like references refer to like parts, and in which: Figure 1A is a flow diagram of the method according to an example; Figure 1B is a schematic representation of the input / output process of the method according to an example; Figure 2 is a pipeline process for monitoring the attention of a subject using of a Multiview model according to an example; Figure 3 is pipeline process for detecting and identifying an object using of a Multiview model according to another example; Figure 4 is pipeline process for detecting and identifying an electrical connector using of a Multiview model according to a further example; Figure 5 is a block diagram of the two-head architecture Multiview model according to an example; Figure 6 is a schematic representation of the neural network architecture according to an example; Figure 7 is a schematic representation of a system according to an example; Figure 8A is a representation of the arrangement of the viewpoints according to an example; Figure 8B is a representation of the arrangement of the viewpoints according to another example; and Figure 8C is a representation of the arrangement of the viewpoints according to a further example. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS A computer implemented method and a system for identifying a body or a pose of a body using a machine learning algorithm based upon a neural network architecture are disclosed. In the following description, a number of specific details are presented in order to provide a thorough understanding of the examples of the disclosure. It will be apparent however to a person skilled in the art that these specific details need not be employed in order to practice the examples of the disclosure. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity in presenting the examples. Figure 1A illustrates the main steps of the method 100 for identifying a body 9 or a pose of a body 9. At step S101, the method 100 comprises providing input data 1. The input data 1 represent at least one 2D image of the body 9 taken from a view point 4. It is noted that the image can be taken using a low resolution camera positioned at a particular angle of view with respect to the body. For example, the image can be a front, a lateral or a rear representation of the body 9. At the same time, the image can be a top or a bottom representation of the body 9. At step S102, a machine learning model is applied to the input data 1. In other words, the image is processed and fed into a neural network following a 2-head architecture. In order to reduce noise and enhance the data quality, the input data 1 can be pre-filtered or pre-processed. Finally, at step S103, a digital representation of the body 9 or of the pose of the body 9 is obtained as output data 2. The body 9, or its pose, can be detected and identified. These process steps are schematically illustrated in figure 1B. In particular, the machine learning model includes the neural network architecture 2 that incorporates a two-head architecture Multiview model. The two-head architecture Multiview model treats the view point 4 of the image and a category 5 of the body 9, or a category 5 of the pose, as two independent variables, for extracting features of the body 9 from different view points 4. By using a neural network architecture 2 which factors the class (category 5) of the body 9, and / or of the pose of the body 9, and the viewing angle (view point 4) of the camera taking the 2D image of the body 9, and / or of the pose of the body 9, it is possible to make images taken from different viewpoints as a single entity, thereby leading to a high degree of scalability reducing inference speeds. Also, since a Multiview model is used to extract the features from different viewing point, the costs of specialized infrastructure are reduced since the images can be captured from anywhere. According to one application of the method 100, the body 9 can be a human person. In this case, the method 100 can advantageously be used for monitoring the attention of the human person. Attention monitoring methods / systems gain importance in multiple environments to control subjects’ attention spans and general behavior. This is highly important to ensure complete alertness and efficiency in work environments and critical spaces like factory work floors and driver cabins. Attention monitoring methods / systems also aid in psychological research and counselling. The known attention monitoring methods / systems are highly focused on a specific region of interest of the person, namely facial features. Since the facial features are highly sensitive, it is necessary the employment of high resolution cameras that are placed to be completely focused on the face of the subject. This also creates a need for establishing specialized camera infrastructure for camera monitoring, thereby leading to higher costs. Moreover, the results provided by known methods are less reliable since the localization of the focus on the face of the person is not the only monitoring attribute contributing to the attention span of the subject. The use of a Multiview model ensures that the complete body pose (and not only the face poses) can be captured and analyzed from any angle, hence allowing the use of existing cameras in the environment and making the results of the method 100 according to the present disclosure more reliable. In one example, in order to enhance the method’s reliability, when the body 9 is a human person, the category 5 of the pose identified as one of the two independent variables can be obtained by a skeleton point detection applied to the input data 1. In particular, the addition of a skeletal point detection module improves the pose inferencing with higher efficiency. In another example, the category 5 of the pose identified is used to compute an attention score representing an attention of the human person. The generation of an attention score for analysis can be done by linking detected poses with the expected poses with respect to the activity in concern. Accordingly, the method comprises assigning to the category 5 of the pose identified a weight score according to an activity of the pose. For example, stretching of body or yawning is having a negative weight when it comes to critical activities. In activities like learning or education, raising of hand, is considered to have positive score. An overall attention score can be generated for a subject or group of subjects. The pipeline process 200 for monitoring the attention of a subject using of a Multiview model is shown in figure 2. A stream of 2D images of a person taken form different view points is provided as input data 1. A Multiview Al model, created by training a machine learning algorithm on a training image dataset, is applied. Meaningful and informative characteristics from the images are extracted (feature extraction). Differently form prior art, the extraction is not limited to facial features, the application considers. On the other hand, the overall poses of the body are considered, thereby leading to more accurate attention score generation. It is noted that the employment of high-resolution cameras (e.g. 3D cameras) is not necessary since it is not required the extraction of specific sensitive features, but rather an estimation of the overall poses. Accordingly, lower resolution cameras (e.g. 2D cameras) can be used instead. According to another application of the method 100, the body 9 can be a manufactured object and / or a part of said manufactured object. In this case, the method 100 can advantageously be used for identifying objects used in larger assemblies of a manufactured system. It is noted that with the term “identification” is intended here the extraction of all the features of the object that will be required by a person to map this component to its part number. The method 100 collectively uses all the extracted features of the object or component to provide the right part number and possibly redirect the user to the additional resources for the object / component, offered by the manufacturer. This identification method 100 further facilitates in offering suggestions for potential replacements currently available in the market for the existing manufactured objects. The present method 100 can be considered as a computer vision pipeline, wherein the class (category) of the objects and the viewing point of the objects in the image are identified. Accordingly, features of the manufactures objects are extracted, which are used to identify the part number of the same from the Consumer Goods Manufacturer catalogue. The identification of the object and the viewing point of the image by the neural network architecture allow the neural network to identify the objects irrespective of the view points 4. The neural network architecture is trained with images of the object from a predefined set of viewpoints. These images are rendered from the 3D CAD files of the object using scripting. By employing rendered images, it makes the learning process both fast and automated. In one example, when the body 9 is a manufactured object and / or a part of said manufactured object, the category of the body 9 is an ID number of the object defined by a manufacturer of the manufactured object and / or part of the manufactured object. In particular, the manufactured object can be an electrical connector. With reference to figure 3, the pipeline process 300 for detecting and identifying an object as an electrical connector is shown. A stream of 2D images of an electrical connector is taken form different view points and is provided as input data 1. A Multiview Al model, created by training a machine learning algorithm on a training image dataset, is applied. Specifically, the images of the connector are processed and fed into a neural network following a two-head architecture. The neural network identifies the category of the connector and the viewing point the image has been taken. The process 300 further comprises verifying the suitability of the view point for extracting the features of the manufactured object and / or part of the manufactured object, in this case of the connector and / or part of the connector. In one example, features are extracted based on the category 5 of the manufactured object and / or part of the manufactured object (e.g. connector). The features can be mapped with a database to retrieve the ID number. Figure 4 shows the pipeline process 400 for detecting and identifying the part number identification of an electrical connector. Since some features are available only from a view point (e.g. from a top view) and few are available from another view point 4 (e.g. from a bottom view), the architecture 2 has a first step to identify and map different views. From the respective views the model collects classified design features and identifiers to predict the class of the connectors. From the family of the connector based on the bottom view features, the variants of the product are identified to assign the correct part number from the list of products. With the present method (100), it is possible to precisely map the part number for the manufactured object, rather than just identifying the category of the object. Also, the two-head neural network architecture allows for high degree of scalability and a large number of categories. Furthermore, by extracting features of the manufactured objects it is possible to transfer knowledge to other applications such as in the field of the robotics. As shown in figure 5, the neural network architecture 2 comprises an extractor backbone element 6 for learning and extracting relevant features from the input data 1. In this case, input data are represented by 2D images of a 3D object, e.g. on an electrical connector. The backbone element 6 consists of multiple layers of neural networks that are stacked together to form a deep network architecture 2 A detail of the neural network architecture 2 is illustrated in figure 6. By considering the objects category 5 and viewing point 4 as independent variables, the neural network architecture 2 enables the network to learn all possible perspectives of the object as a single entity. In contrast to known neural network architectures, the present one permits linear scaling of the number of parameters to bigger applications. The backbone element 6 has a number of nodes in the final layer equal to m+n, wherein m is the number of categories 5 of the manufactured object and / or part of the manufactured object and n is a number of view points 4. This is the perfect choice for highly scalable applications with many categories since the number of parameters scales linearly. The neural network architecture 2 has been extended for object detection and image segmentation which outputs coordinates to identify the region of interest. The architecture 2 has been tested with different feature extractor backbone 6, including resnet50, efficientnetv2. X and Y are the number of nodes in the final layer which correspond to number of classes (categories 5) and views (view points 4) respectively. In case of the ModelNet40, X = 40 since there are 40 classes and Y = 20 since images are considered at fixed intervals. As shown in the figures, the extractor backbone element 6 provides a total loss equal to the sum of a category loss and a view point loss. In particular, to minimize the error during training, the total loss can be backpropagated into the network architecture 2. According to an example, the input data 1 represent at least two 2D images taken by one camera at fixed time interval or the input data 1 represent a plurality of 2D images taken by a plurality of cameras, positioned at fixed distances. The second arrangement is illustrated in figure 7. Here, a plurality of 2D cameras 8 are positioned affixed distances for taking images of the body 9. It is noted that figure 7 shows eight cameras 8; however, the present system 7 and method are not limited to this number. In one example, the 2D cameras 8 are positioned at equidistant positions along a circumference. Assuming one axis as the rotation axis, the set of view points 4 can be positioned affixed intervals, e.g. at 30 degrees intervals (Fig. 8A). In alternative, a single camera 8 is moved along a circumference path and the images are captured at predefined time intervals. According to another example, all the view points 4 are equally spaced from the neighboring points and the set of view points 4 can be positioned at the vertices of a Dodecahedron (Fig. 8B). To obtain a perfectly fixed top and bottom view of the object 9, the set of view points 4 can be positioned at the vertices of a Icosidodecahedron (Fig. 8C). It is noted that tests have been performed to check the efficiency and reliability of the method 100 and system 7 using a variety of images of manufactured objects (e.g. electrical connectors) from different categories. The pipeline process showed to consistently work with a confidence level of 65%. The neural network architecture 2 has been tested on ModelNet40 dataset which includes 40 categories of manually aligned CAD models of common objects. The CAD models are converted into images from 20 fixed viewpoints. The model is evaluated on this data and results in 98% view accuracy and 94% classification accuracy. The architecture 2 has also been tested on CAD model of PCB Connectors which was converted to 30 fixed viewing point images. The architecture 2 got an accuracy of 100% for classification and 96% on identifying the views. Although a variety of techniques and examples of such techniques have been described herein, these are provided by way of example only and many variations and modifications on such examples will be apparent to the skilled person and fall within the spirit and scope of the present invention, which is defined by the appended claims and their equivalents.

Claims

1. Computer implemented method (100) for identifying a body (9) or a pose of a body (9) using a machine learning algorithm based upon a neural network architecture (2), the method (100) comprising:providing (S101) input data (1) representing at least one 2D image of the body (9) taken from a view point (4);applying (S102) a machine learning model to the input data (1); andobtaining (S103) a digital representation of the body (9) or of the pose of the body (9) as output data (2), wherein the machine learning model includes a neural network architecture (2) that incorporates a two-head architecture Multiview model treating the view point (4) of the image and a category (5) of the body (9), or a category (5) of the pose, as two independent variables, for extracting features of the body (9) from different view points (4).

2. Computer implemented method (100) according to claim 1, wherein the body (9) is a human person and the category (5) of the pose identified as one of the two independent variables is obtained by a skeleton point detection applied to the input data (1).

3. Computer implemented method (100) according to claim 2, wherein the category (5) of the pose identified is used to compute an attention score representing an attention of the human person.

4. Computer implemented method (100) according to any one of claims 2 to 3, further comprising assigning to the category (5) of the pose identified a weight score according to an activity of the pose.

5. Computer implemented method (100) according to claim 1, wherein the body (9) is a manufactured object and / or a part of said manufactured object and the category ofthe body (9) is an ID number of the object defined by a manufacturer of the manufactured object and / or part of the manufactured object.

6. Computer implemented method (100) according to claim 5, wherein the manufactured object is an electrical connector.

7. Computer implemented method (100) according to any one of claims 5 to 6, further comprising verifying (S104) the suitability of the view point for extracting the features of the manufactured object and / or part of the manufactured object.

8. Computer implemented method (100) according to any one of claims 5 to 7, wherein features are extracted based on the category (5) of the manufactured object and / or part of the manufactured object and the method further comprises mapping (S105) said features with a database to retrieve the ID number.

9. Computer implemented method (100) according to any one of claims 5 to 8, wherein the neural network architecture (2) comprises an extractor backbone element (6) having a number of nodes in the final layer equal to m+n, wherein m is the number of categories (5) of the manufactured object and / or part of the manufactured object and n is a number of view points (4).

10. Computer implemented method (100) according to claim 9, wherein the extractor backbone element (6) provides a total loss equal to the sum of a category loss and a view point loss.

11. Computer implemented method (100) according to claim 10, wherein the total loss is backpropagated into the network architecture (2).12 Computer implemented method (100) according to any one of claims 1 to 11, wherein the input data (1) represent at least two 2D images taken by one camera at fixed time interval or the input data (1) represent a plurality of 2D images taken by a plurality of cameras, positioned at fixed distances.

13. Computer program, comprising instructions which, when the program is executed by a computing device, cause the computing device to carry out the computer-implemented method (100) according to any one of the claims 1 to 12.5 14. Computer readable medium comprising a computer program for carrying out themethod (100) according to one of claims 1 to 12.

15. System (7) for identifying a body (9) or a pose of a body (9) using a machine learning algorithm based upon a neural network architecture (2) using the method (100) 10 according to any one of the claims 1 to 12, the system (7) comprising a plurality of 2D cameras (8) for monitoring the body (9) and taking at least two 2D images by one camera at fixed time interval or taking a plurality of 2D images by a plurality of cameras, positioned at fixed distances.

Images