A positioning system

Abstract

A computer-implemented method of guiding positioning, comprises the steps of: receiving at a computing device first sensor data, such as camera or LIDAR data, of a three-dimensional first object, such

Description

The present application relates to a positioning system, and in particular concerns the system for positioning a sensor with respect to an object to be sensed. A device and method for assisting users with guided self-examination of their breasts is disclosed in GB2614912. This document describes a hand-held acoustic sensing device, which is held by a user and moved across the skin of their breasts as part of a guided self-examination process. In the method described in GB2614912, a representation of the user’s torso is presented on, for example, the screen of a mobile phone, divided into a number of sub-regions, which can be presented as a grid of rectangles or squares. The system guides the user to move the acoustic scanning device to each square in turn, to ensure that a full examination is completed. In the system of GB2614912, the position of the acoustic scanning device is determined by the movement of a roller ball, which is provided as part of the scanning device, as well as by inertial sensors such as one or more accelerometers and / or a magnetometer. It is an object of the invention to provide an improved system and method for guiding an examination of this kind. Accordingly, one aspect of the present invention provides a computer implemented method according to claim 1. A further aspect of the invention provides a computer programme according to claim 21. Another aspect of the invention provides a system according to claim 23. Preferred features of the invention are set out in the dependent claims. In order that the invention may be more readily understood, embodiments thereof will now be described, by way of example, with reference to the accompanying figures, in which: Figures 1a to 1f show a sensor device suitable for use with the present invention; Figure 2 shows point cloud data representing the front of a user’s torso; Figure 3 shows a depth map derived from the data shown in figure 2; Figure 4 shows the creation of a mask, based on the depth map of figure 3; Figure 5 shows the application of the mask to the data shown in figure 2; Figures 6 and 7 are flow diagrams showing important features of methods embodying the invention; Figure 8 shows an example of a shape of part of a sensor device in accordance with the invention. Figure 9 shows an example of a shape that can be displayed on a sensor device in accordance with the invention; Figures 10 to 13 show views that may be presented to a user during a method embodying the invention; and Figure 14 shows major components of a system embodying the present invention. When carrying out a method in accordance with the invention, a user will require an external computing device which has a camera or LiDAR sensor, which can be directed towards the user’s torso. It is envisaged that the most convenient and widely available type of external computing device will be a mobile phone, and the discussion below will use a mobile phone as the external computing device, but the skilled readerwill understand how any othersuitable kind of device, such as a desktop computer with a webcam, or a laptop with a built-in camera, may be used with the invention. As the method is carried out the external computing device will operate through a computer programme that controls one or more components and processes of the external computing device. In the case of a mobile phone, the programme may take the form of a dedicated application (i.e. “app”). However, the programme may take any suitable form. Steps of a method embodying the invention will now be described. In a first step, a LiDAR sensor or camera of the external computing device is used to generate a 3D model, representing the front of the user’s torso. A user directs the LiDAR sensor or camera of the external computing device at the front of their torso. It is preferred that as this occurs a representation of the data gathered by the camera or LiDAR sensor is displayed to the user on a screen of the external computing device. This will assist the user in directing the camera or LiDAR sensor accurately, and / or in moving to be correctly within the field of view of the camera or LiDAR sensor. An outline may be presented on a screen of the device, in combination with a representation of data gathered by the LiDAR sensor or camera, such that when the user’s torso is within, or aligned with, the outline, the user is correctly positioned within the field of view of the camera or LiDAR sensor, and is at an appropriate distance from the camera or LiDAR sensor. The representation of the data from the camera or LiDAR sensor may be blurred, frosted or otherwise altered for the purposes of modesty as this process occurs. When the user is correctly positioned with respect to the LiDAR sensor or camera, The LiDAR sensor or camera is activated to capture data relating to the front of the user’s torso. Where the external computing device has both a camera and a LiDAR sensor, images from the camera may be used to help the user align their torso correctly, and data from the LiDAR sensor is then captured for the purposes of generating the depth map. As a skilled reader will understand, a LiDAR sensor emits a laser, and measures the time for the reflected laser light to return to the sensor. In embodiments of the invention the LiDAR sensor will scan in multiple directions, in a stepped fashion, generating a grid of points, each with a determined distance from the LiDAR sensor. Preferably this raw data is converted into point cloud data. Figure 2 shows a representation of point cloud data 7 of the captured data representing the front of the user’s torso. As a further step, the remaining parts of the point cloud data may be transformed into a 3D mesh. As the skilled reader will understand, this involves interpolating between the points of the point cloud data, to connect the points and thus construct a 3-dimensional approximation of the contours of the front of the user’s torso. However, in other embodiments the point cloud data is not converted into a 3D mesh. Where the external computing device has a camera, but not a LiDAR sensor, the camera is used instead. The skilled reader will appreciate how data from a camera may be used to generate a depth map. For instance, a neural radiance field (NeRF) technique may be utilised, in which camera images of the user’s torso are captured from different angles, and a 3D model is built up from these images using a trained neural network. If a technique such as this is used, the external computing device may prompt the user to position the camera to capture an appropriate number of images from different angles, and the skilled reader will understand how this may be achieved. In preferred embodiments of the invention, at least some, and preferably all or substantially all, of the dimensions of the 3D model are proportionate to those of the user’s torso. As the skilled reader will understand, this means that the relative sizes and shapes of the features that appear in the 3D model are the same or substantially the same as those of the corresponding features of the user’s torso. The 3D model that has been created is then converted into a 2D depth map. As the skilled reader will understand, the depth map comprises a two-dimensional grid of points, and / or one or more planes or other surfaces, with each point and / or surface being associated with a depth, or height, with respect to a plane. Figure 3 shows a representation of a 2D depth map 8, generated using the data shown in figure 2. In generating the 2D depth map, a depth buffer (orZ-buffer) may be generated, as will be understood by the skilled reader. The depth scale may be normalised, and differences in height may be exaggerated, to assist in later processing operations. One benefit of converting the 3D model into a 2D depth map is that the depth map can readily be analysed in ways that will assist in later analysis. In preferred embodiments, the external computing device is trained to recognise features of a torso in the depth map. For instance, a machine learning (ML) process may be trained on data representing a number of torsos, to recognise features such as shoulders, breasts, nipples, a navel, a waist and hips. The ML system may be used to analyse the data obtained from the LiDAR sensor or camera, to identify some or all of these features in the data. Such a ML system may comprise one or more artificial neural networks, for example, trained on labelled examples of images including one or more of the recognisable features. In embodiments of the technology, the ML process involves training and use of a convolutional artificial neural network (ANN), suitable for image processing. In some embodiments, the ANN may comprise between 10 and 150 layers. In preferable embodiments, the ANN comprises between 23 and 100 layers. The input to the ANN represents the 3D model and / or its data points, and in embodiments, this is presented as a 480 by 680 pixel image of the patients’ torsos. For example, the model data may be input to the ANN as the converted 2D depth map, having a size of 480 by 680 pixels. Generally, an image size of between 50 by 50 pixels, and 1500 by 2000 pixels, is deemed suitable, and the aspect ratio of the image need not be limited to 480 to 680, and it should be understood that a variety of shapes and sizes of images are suitable for this purpose. In an example, a training set of such images was collected from 120 patients. Portions of the images, or locations on the images, were labelled with feature labels as mentioned above (i.e., shoulders, breasts, nipples, etc.), fortraining purposes. The ANN was trained using standard known training methods, on this training set. The ANN is trained to output one or more binary image maps, each corresponding to one or more features. In this way, the trained ANN is used to output binary image maps representing the portions of the image corresponding to the features trained; i.e., to identify the positions and / or presence of those features in the input images. In preferred embodiments, the depth map is analysed, and parts of the data that do not represent relevant regions of the user’s body are identified. These parts of the data could represent, for example, the user’s neck, face, lower arms, and regions below the user’s waist. This information can then be used to exclude the corresponding parts of the data on which the 3D model is based. In preferred embodiments, when the depth map has been created, a corresponding binary mask may be created. A binary mask 9 is shown in figure 4, alongside a corresponding depth map 10. In the example shown in this figure, regions corresponding to desired regions of the torso are represented in one colour (here, white), while other regions are represented in a second colour (here, black). The other regions can comprise parts of the depth map that do not represent the torso at all, and also regions of the depth map that represent regions of the torso that are not relevant or not needed for analysis. As a next step, a polygon is generated from the binary mask, with the polygon following the shape of the binary mask. The polygon may then be cropped from the data on which the 3D model is based, for example original point cloud data, to leave data representing only the desired parts of the torso. This is represented schematically in figure 5. For example, data relating to the user’s breasts, and also surrounding areas such as the user’s shoulders, armpits and midriff, may be retained in the 3D model. This results in data covering the user’s breast tissue, but also other surround features which will assist in analysis during later stages, as will be explained below. However, other regions of the user’s torso or body that are not required for analysis are preferably removed from the data on which the 3D model is based. The skilled reader will understand that the resulting data represents a selected set of features of the user’s torso, but no further features, to minimise the size of the data that needs to be processed and manipulated. In a further step of the method, at least some parts of the 2D depth map are subdivided into a number of sub-regions. In some examples, only the parts of the 2D depth map that correspond to the user’s breast tissue are divided into sub-regions. As discussed above, the parts of the data representing the breast tissue can be identified by a ML system. However, in other examples further regions are also divided into sub-regions. In some examples, all or substantially all of parts of the 2D depth map are divided into sub-regions. For instance, in a preferred embodiment, all of the front of the torso is divided into sub-regions (as shown in figures 9 to 12, discussed in more detail below), with only some of these sub-regions being used in later analysis. For example, the sub-regions between the user’s clavicle and the bottom of the user’s ribcage (preferably across the width of the torso) may be used for analysis, with the other sub-regions not being used for analysis. In preferred embodiments of the invention, the sub-regions are generally rectangular, having four sides. The square or rectangular shape of the sub-regions is preferably as projected onto the surface of the depth map, and so the sub-regions may not appear to be square or rectangular in an image based on the data. The sub-regions are preferably contiguous or substantially contiguous with each other, to form a grid of sub-regions that cover all or substantially all of the parts of the user’s breast tissue (and optionally some or all of the other parts of the user’s anterior chest) that are represented in the 2D depth map. The skilled reader will note that, for users with larger breasts, there will be a region of the underside of each breast that is not visible from a front view. Techniques for addressing this will be explained in more detail below. However, if these hidden regions are not represented in the 2D depth map, they will also not be covered by the sub-regions. In some embodiments, the size of the sub-regions may vary depending upon the dimensions of the user’s torso. For instance, as discussed above, distinctive features of the user’s torso may be identified in the depth map, such as the user’s shoulders. Dimensions of the user’s torso, derived from the spacing of these distinctive features, (such as the distance between the user’s shoulders) may influence the size of the sub-regions. Alternatively the area of the skin of the user’s breasts may be determined or estimated, with the size of the sub-regions being influenced by this area. In such embodiments, the number of sub-regions that are generated may be the same or substantially the same for all users, with the size of some or all of the sub-regions being adjusted in dependence upon dimensions of the user’s torso. In other embodiments, the size of the sub-regions may be constant or substantially constant, regardless of the dimension of the user’s torso. Where a user has, for instance, a relatively large distance between her shoulders, or relatively large breasts, a greater number of sub-regions may be generated. The skilled reader will readily appreciate how each of these processes may be implemented. The sub-regions that have been determined in this way can then be mapped onto the 3D model. The skilled reader will appreciate that converting the 3D model into a 2D depth map will allow improved and / or more efficient analysis of the torso, identification of the most relevant parts of the data, and / or segmentation of parts of the torso into sub-regions, using ML or classical method such as computer vision methods. The results of this analysis can then be applied to the 3D model for use in later stages of the method. Figure 7 shows important steps in methods of data processing and recognition embodying the present invention. In a first step 7a, an unprocessed 3D PLY model (i.e. Polygon file format model), generated from LiDAR output, is transformed into a smooth 2D depth map, which preferably has exaggerated features such as depth. In a second step 7b, a trained convolutional neural network is used to segment an area of interest of the model, and to produce a binary mask for this area. In a further step 7c, a 2D polygon is generated from the binary mask, and used to crop its projection onto the 3D model. The listing of the steps as 7a-7c above, and their presentation in a particular order in figure 7, does not signify that it is essential that all of the steps are carried out in the order in which they are presented. During subsequent steps of the method, the front of the user’s torso will be maintained within the field of view of the LiDAR sensor and / or camera of the external computing device. Movement of the user’s torso during these subsequent steps is tracked. One technique for this is to identify distinctive points of the front of the user’s torso, for instance, the user’s left and right shoulders, left and right nipples, and / or navel (as discussed above, these points may be identified, for instance in the depth map, by a trained ML system). The LiDAR sensor and / or camera will monitor the positions and / or distance from the sensor of these distinctive points, and this monitoring may be continuous or substantially continuous, or take place at intervals, which may be regular intervals. An examination process will then take place, during a time period which is subsequent to the time over which the data is captured to create the 3D model of the user’s torso. A representation of the front of the user’s torso will preferably be shown to the user on a screen of the external computing device, as the examination proceeds. This representation is based on the 3D model that was previously created. It is likely that the user’s torso will change position and / or orientation during the examination. These changes will be tracked, for instance through monitoring of the distinctive points. The representation shown on the screen may change to reflect the detected movement of the user. In this way, a digital representation of the front of the user’s torso is created and displayed to the user. The digital representation preferably moves and / or changes, preferably in real-time or substantially real-time, to mirror the user’s movements. The digital representation is preferably a 3D representation. The sub-regions identified during analysis of the 2D depth map, as discussed above, are preferably displayed on the representation of the front of the user’s torso. In preferred embodiments, the representation shown to the user on the screen does not include any live camera data, and shows the digital representation of the torso, but no camera images of the torso. However, this is not essential, and in other embodiments the digital representation may, for example, be shown superimposed upon camera images of the user’s real torso. In subsequent steps of the method the user is directed to carry out a self-examination, covering each of the sub-regions that have been defined. To do this, an object that will be used to carry out the self-examination is presented in the field of view of the camera and / or LiDAR sensor. In a first example, the self-examination is carried out using a part of the user’s hand. In this example, the tip of the user’s index finger is used. The user will be prompted to hold their hand in the field of view of the camera or LiDAR sensor. The shape of the user’s hand will be recognised in the data gathered (for instance, by a trained ML system, as discussed above) and the position of the tip of the user’s index figure is identified. The skilled reader will be readily aware of techniques by which this may be done. The user may be asked or encouraged to apply a different or distinctive shade of nail polish, or apply a distinctive item such as a sticker or reflector, on the tip of her index finger to assist in this step of analysis. Positional data, which may be 2D, for the fingertip is generated by the camera or LiDAR sensor. This positional data may be related, preferably by 3D trigonometric methods, to the position of the user’s torso, to calculate where on, or in front of, the user’s torso the fingertip is located. The tip of the user’s index finger may be represented in the image on the screen, for instance as a dot. The position of this fingertip with respect to the user’s torso will therefore be clearly displayed to the user. In preferred embodiments, the rest of the user’s hand and arm will not be displayed in the image. The user will therefore see an unobstructed or substantially unobstructed representation of her torso, along with a dot or other representation of the tip of their index finger. The user is then directed to place the tip of their index figure in a first target one of the sub-regions. This may be done in any suitable way, for instance by highlighting the first target sub-region in a particular colour, such as blue. The user, guided by the image on the screen of the external computing device, moves the tip of their index finger to be within the first sub-region, and carries out a palpation within this first subregion fora predetermined length of time. The external computing device preferably makes a determination that the tip of the user’s index finger is within the first sub-region. This may be done, for example, by determining that the fingertip is against or close to the surface of the user’s torso, and within the first sub-region, or that the fingertip is aligned with the first sub-region, from the point of view of the camera or LiDAR sensor. The palpation is preferably carried out for a predetermined length of time, and in one example this length of time may be 4 seconds (although any other suitable length of time may be used). In one example, when the tip of the user’s index finger is determined to be within the first sub-region, the length of time for which the user’s fingertip remains within the sub-region is counted, and once the predetermined length of time has elapsed, an indication is given to the user. This may take any suitable form, for instance a change in colour of the sub-region in the representation shown on the screen, the presentation of a symbol or icon (such as a tick or “thumbs up”), and / or haptic or audio feedback from the external computing device, or from a sensor, if one is being used, as discussed below. During each palpation, guidance may be provided on the screen (or in another way, such as a voice announcement) instructing the user in how to move the tip of their index finger to perform the palpation. For instance, the user may be encouraged to move the tip of their finger in a circular motion as the palpation is performed. In one example, a symbol representing a circular motion may be presented on the screen, and this symbol could take the form of a line which forms almost a complete circle, with an arrow head on one of the free ends of the line. This symbol may rotate. In preferred embodiments, a symbol of this kind may be shown with time information also presented (for instance, with the time information being positioned within or alongside the symbol indicating circular motion). For example, a number which counts down from 4 seconds may be presented within the circular symbol. This will provide effective and intuitive guidance to a user to perform circular motions, for a suitable length of time, as the palpation within each sub-region. Once this is done, the user is directed to move the tip of their index finger to a second sub-region, which is preferably adjacent the first one. Sub-regions in which a palpation has already been done may be marked or shaded in a distinctive colour, such as green. In this way, the progress of the user through the self-examination process will be readily displayed and easy to track. Figures 10 to 13 show examples of representations that could be shown to a user as the examination proceeds. Figure 10 shows a representation 12 of the user’s torso, divided into a number of sub-regions 13. A palpation has been carried out in six of these sub-regions 13a already, and these six sub-regions 13a are highlighted in a different colour. Sub-regions 13 in which a palpation has not yet been performed are shown in black. The position of the user’s fingertip is presented as a dot 14. In figure 11, a further sub-region 13b is highlighted in a colour which is different from the one representing the regions in which a palpation has already been carried out. In figure 12, the user has moved their finger to the further sub-region 13b, and the dot 14 is therefore in this further sub-region 13b. Once a palpation has been carried out in this further sub-region 13b, the further sub-region 13b is shown in the same colour as the previous six sub-regions 13a in which a palpation had already been carried out. This is shown in figure 13. As the user performs palpations at each sub-region, a record of the results of the palpations is formed. In other examples, the user is not guided as to an order in which the sub-regions should be examined. In such examples, the user may carry out a palpation in the sub-regions in any convenient order. The system may detect when the tip of the user’s index finger has been within any sub-region fora particular length of time (for instance, 4 seconds), and assume that a palpation in that sub-region has been carried out. In a preferred embodiment, as the palpation takes place at each of the sub-regions, the user is given an opportunity to indicate that the palpation has revealed something that is potentially abnormal, such as a lump or change in density. If the user indicates that a potential abnormality has been discovered, the user may be prompted to input further information relating to the potential abnormality, and this may take any suitable form, such as a menu of options or a free text field. For example, the user may be prompted to move sliders or one or more other interfaces to input information about the hardness, shape, and / or movement of any feature found, as well as the skin texture and whether the feature is painful. In preferred embodiments, as the palpation takes place at each sub-region, the user has an option to interact with the external computing device to indicate that a potential abnormality has been found, for instance by touching the screen or through voice activation. However, if no such input is received from the user, it is assumed that no potential abnormality has been found in that subregion. In this way, unless and until a potential abnormality is found, the user can proceed with the self-examination without needing to give any direct input to the external computing device. As the skilled reader will appreciate, in embodiments of the invention a digital representation of the front of the user’s torso is presented on a screen of the external computing device, and the position of the object used forexamination (in this example, the tip of the user’s index figure) is also represented on the screen. The user is therefore able to use the object to interact with the digital representation of the torso. During this process the position and orientation of the torso may also change, preferably in real time, to mirror movements of the user’s torso as the examination proceeds. The skilled reader will understand that this provides a straightforward and effective way to interact with the digital representation, allowing an easy and intuitive self-examination process to be guided and performed. In the course of carrying out the examination, the user’s hand or arm may obscure one of the distinctive points of her torso that is being tracked to monitor the position and orientation of the torso. It is envisaged that, if this occurs, enough further distinctive points will still be uncovered and available to be tracked. In further embodiments of the invention, instead of a part of the user’s hand a sensor device may be used to perform the examination. With reference firstly to figures 1A to 1F, a sensor device 1 suitable for use with the present invention is shown. The sensor device 1 comprises a main body 2 and a handle 3, which are connected by a stem 4. In the embodiment shown, the handle 3 is wider than the stem 4. In use, it is intended that a user may grip the stem 4 between two fingers, for instance the index and middle fingers, to hold the device 1 with one hand in a comfortable and functional manner. The stem 4 may be formed from a durable and flexible material such as an elastomer (for examiner silicone rubber) to allow a degree of relative movement between the body 2 and the handle 3. In preferred embodiments the handle 3 has an outer surface 5, that extends generally away from the body 2, with all, substantially all, or a significant portion of the surface 5 being flat or substantially flat. The reasons for this will become clear from the discussion below. The main body 2 has a contact surface 6 formed on a lower side thereof, i.e. a side generally opposite the side on which the handle 3 is formed. The contact surface 6 is adapted to be placed against the user’s skin during use. The contact surface 6 is preferably flat or substantially flat, although this need not be the case. In the example shown, the contact surface 6 is formed at an angle with respect to a plane (indicated generally by the line A-A in figure 1c) which is perpendicular to the axis of the stem 4. It has been found that, for conducting examinations of one’s breasts, the formation of an angle of this kind is preferable. In preferred embodiments, the angle between the plane and the contact surface 6 may be around 10-12°, although this is not essential. The device shown in the figures does not include a roller ball, or a similar device such as a linear roller. It is envisaged that the provision of a roller ball will be unnecessary, and can be omitted to save on cost, complexity and weight. However, in other embodiments a roller ball or similar component may be included in the sensor device 1, to make it easier for the user to move the sensor device across her skin, particularly if pressure is being applied to the sensor device as it is moved. However, it is preferred that, in such embodiments, the roller ball or similar component plays no part in tracking the motion of the sensor device. The sensor device 1 includes an acoustic generator and an acoustic sensor (not shown in the accompanying figures). In use, the acoustic generator generates an acoustic signal which is emitted from the device 1, preferably through the contact surface 6. The acoustic sensor receives sound emitted by the acoustic generator, after the sound has been reflected by an object, such as part of the user’s body. As the acoustic signal produced by the acoustic generator travels through the body, its frequency may change as a result of changes in soft tissue density. Analysis of the frequency of the received sound, detected overtime, may indicate the presence of developing tissue abnormalities. The generation and receipt of acoustic signals, by a respective acoustic generator and acoustic sensor, or through a unitary acoustic transducer, is known perse and will not be discussed in detail in this application. Details of types of acoustic generator and acoustic sensor that can be used in conjunction with this invention are described in more detail in GB2614912. In preferred embodiments of the invention, the acoustic generator produces an ultrasonic acoustic signal, i.e. a signal having a frequency greater than about 20 kH. The use of ultrasonic frequencies is well-established in many fields, and suitable generators and sensors may be readily obtained. However, frequencies below the ultrasound range may also be used, and this may reduce the cost of the generator component. In alternative embodiments the acoustic generator produces one or more frequencies which are preferably within the range of 300 Hz to 19 kHz, and which may be within the range of 600 Hz to 6 kHz. The use of any suitable frequencies is encompassed within the scope of the invention. In preferred embodiments of the invention, the transducer is a micromachined system (MEMS). In yet more preferred embodiments, the transducer is of a CMUT design or a PMUT design. Transducers of this kind can have beneficial miniaturisation and accuracy, at a reasonable cost. The sensor device 1 further comprises a microcontroller, configured to control operation of the acoustic generator and acoustic sensor, and a power source to provide power to the components of the sensing device. The sensing device may further comprise components such as an on-off switch, and an indicator light, such as an LED, to indicate to a user that the device is operational. The sensor device 1 preferably comprises a communication module, such as a Bluetooth transceiver module, configured for wireless communication with the external computing device. Any suitable form of wireless communication between the sensing device and the external computing device may be employed. Once again, further details of the components of one possible sensor device that could be used in conjunction with this mention are shown in detail in GB2614912. In embodiments which include the sensor device 1, the user is preferably directed by the system to hold the sensor device 1. The user may hold the sensor device 1 so that it is within the field of view of the LiDAR sensor or camera of the external computing device. The external computing device detects the sensor device 1 in the data gathered by the LiDAR sensor or camera. In the example shown in figures 1A to 1F, the sensor device 1 includes an outer surface 5. If the user holds the sensor device 1 by gripping the stem 4 thereof between two fingers, and places the contact surface 6 against her skin, the outer surface 5 will be visible in a field of view that covers the front of the user’s torso. In preferred embodiments, some or all of the outer surface 5 is detected in the data gathered by the camera and / or LiDAR sensor, to determine the position of the sensor device 1. In some examples, the overall shape of the outer surface 5 may be detected and used to locate the sensor device 1. In the example shown the outer surface 5 has an oval shape, and detection of the shape can be used to establish both the position and orientation of the sensor device 1. The outer surface may take any suitable shape, and may preferably be a shape that has no lines of symmetry, so that the orientation of the outer surface can be unambiguously determined. In further examples, the outer surface 5 may have one or more markers thereon which can be identified in the data gathered by the camera or LiDAR sensor. For instance, the outer surface 5 may have one or more reflectors or other fiducial markers thereon. The reflector or fiducial markers may be arranged in any pattern, and in particular may be arranged in a pattern which allows the orientation of the outer surface 5 to be unambiguously determined. Figure 8 shows one example of an alternative shape for the outer surface 5. In this example the outer surface 5 has the shape of an isosceles triangle, this shape will allow the system to determine the orientation of the sensor device 1 unambiguously. Figure 9 shows a further example in which the outer surface 5 has an oval shape, as in figures 1 a to 1 f, and a symbol 11 which includes a square shape and a five-pointed star is presented on the outer surface 5. The orientation of this symbol 11 in images will once again allow the system to determine the orientation of the sensor device 1 unambiguously. The symbol 11 may be printed or etched directly onto the outer surface 5, or may be provided on a sticker which is applied to the outer surface 5, or may be provided on the outer surface 5 in any other way. The skilled reader will understand that there are many examples of shapes, and other markings (which may optionally take the form of indentations or raised protrusions formed in the outer surface 5). Once the position of the sensor device 1 has been determined, the location of the contact surface 6 (or another suitable part of the sensor device 1, which need not be the part that is detected in the camera or LiDAR data) can be represented in the image displayed to the user on the screen. Once again this position can be represented as a dot, or another suitable icon or symbol. As in the example above, the user’s arm and hand, and other parts of the sensor device, are preferably not displayed or represented on the screen. The screen may display the digital representation of the user’s torso, along with a dot or other symbol representing part of the sensor device, with the digital representation being otherwise unobstructed or substantially unobstructed. The user can then be directed to conduct a self-examination, by moving the sensor device 1 to each of the sub-regions in turn, as discussed above in relation to the earlier example. When the sensor device 1, or a part of the sensor device 1, is positioned within each sub-region, a reading is taken using the sensor device 1. The readings are preferably transmitted to the external computing device. The readings can be analysed to reach a determination as to whether any potential abnormalities have been detected. In doing so, the readings may be compared to a previous set of readings that have been taken from the same user in one or more previous examinations. More detail on how the readings may be processed and analysed is given in GB2614912. Figure 6 shows a flow chart which includes important features of the methods described above. In a first step 6a, a 3D model of elements of a physical object of interest is generated. Subdivisions are rendered on the model, and the size, positioning and / or number of these subdivisions will depend on the real life dimensions of the physical object. In a second step 6b, a camera and / or LiDAR sensor tracks the positions in 3D space of key points on the object of interest, and also of a further object, such as a sensor or other tool, or part of a user’s hand. In a third step 6c, the tracked positions of the key points of the object, and of the further object, are related to the 3D model. This relates to a specific goal that has to do with the subdivisions (such as a self-examination), while the relative movements of the positions are tracked in real life. A representation of the 3D model, and of the further object, is presented visually to the user on a screen. In a fourth step 6d the user is given guidance from the external computing device, for instance to move the sensor or their hand in relation to the subdivisions, until the goal is complete, or until a break is needed. The listing of the steps as 6a-6d above, and their presentation in a particular order, does not signify that it is essential that all of the steps are carried out in the order in which they are presented. Figure 14 illustrates some of the components of the main devices of one embodiment of a system embodying the invention. In embodiments of the technology, an external computing device 15, which may be a mobile phone or tablet, has a camera 16 and / or LiDAR sensor 17. The external computing device 15 further has a display 18, on which images may be presented to a user. The external computing device 15 also has a communication module 19 which may be wireless or wired, for sending and receiving data (e.g., providing WiFi connectivity, a data connection enabling 4G / 5G data transfer, or Bluetooth connectivity). The external computing device 15 has a processor 20, memory device(s) 21, storage device(s) 22, and an input device 23 (which may comprise touch screen technology associated with the display 18) for receiving user input. The external computing device 15 provides a power source 24 which may comprise a power cell which may be removable and / or rechargeable or may comprise a power connector for connection to a mains supply. The sensor 25 includes an acoustic transducer 26, as well as a processor 27 and a power source 28, and also a communication module 29 allowing communication, which is preferably two-way communication, with the external computing device 15. A server 30 may be provided in a remote location, and be in two-way communication with the external computing device 15. The server 30 includes one or more memory devices 24 to store data transmitted by the external computing device 15. The server 30 may perform analysis on data sent from the external computing device 15, and transmit some or all of the results of the analysis to the external computing device 15. The server 30 may also send software updates to the external computing device 15. The above examples are given with generation of a digital representation of a user’s torso, and tracking of an object, such as a tip of a user’s finger, or a sensor device, with respect to the torso. However, the techniques described above may be used to guide examinations and other processes in relation to any other part of a human or animal body, for instance in examinations to check for testicular conditions or soft tissue sarcoma. The use of these techniques is not restricted to checks for growths or abnormalities, and they will find application in guiding or assisting users in other activities such as physiotherapy, massage and acupuncture. These techniques will also be useful in many other applications, such as guided surgery; plastic surgery reconstruction; guided medical learning through extended reality (XR), including augmented reality (AR), virtual reality (VR) and mixed reality (MR); other self guided body procedures, relating to examinations, emergencies or more general wellness; and other guided procedures, on electronics, home appliances, cooking, and so on. As discussed above, where a user has larger breasts there may be a region of the underside of each breast which is hidden from a front view, and hence will not appear in the depth map. Where this is the case, as part of the guided self-examination process, the user will be prompted to perform a palpation of the undersides of her breasts, and to give an input as to whether any potential abnormalities are detected. This can also be done for one or more other regions, such as the user’s armpits. The skilled reader will appreciate that the embodiments described above allow a simple, intuitive and effective guided self-examination, using minimal equipment. When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components. The invention may also broadly consist in the parts, elements, steps, examples and / or features referred to or indicated in the specification individually or collectively in any and all combinations of two or more said parts, elements, steps, examples and / or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features from any other embodiment(s) described herein. Protection may be sought for any features disclosed in any one or more published documents referenced herein in combination with the present disclosure. Although certain example embodiments of the invention have been described, the scope of the appended claims is not intended to be limited solely to these embodiments. The claims are to be construed literally, purposively, and / or to encompass equivalents.

Claims

1. A computer-implemented method, comprising the steps of:receiving at a computing device first sensor data of at least a part of a three-dimensional first object;generating a 3D model of the at least part of the three-dimensional first object from the first sensor data, wherein dimensions of the 3D model are proportionate to those of the first object;receiving second sensor data of the first object and a second object positioned in the vicinity of the first object, during a time period; andpresenting images on a screen, the images including a representation of the first object, during the time period, based on the 3D model, with the position and / or orientation of the representation of the first object being based on the second sensor data, whereinthe images include information indicating the position of the second object with respect to the representation of the first object.

2. A method according to claim 1, further comprising the step of converting the first sensor data into a two-dimensional depth map representing the at least part of the first object.

3. A method according to claim 2, further comprising the steps of:identifying one or more features of the first object in the two-dimensional depth map;tracking the position of the or each feature of the first object in the second sensor data during the time period; andbasing the position and / or orientation of the representation of the first object on the tracked position of the or each feature.

4. A method according to claim 2 or 3, further comprising the steps of:creating a mask, from the two-dimensional depth map, wherein the mask represents a boundary of the first object in the two-dimensional depth map; andapplying the mask to the first sensor data, to remove parts of the sensor data that do not correspond to the first object.

5. A method according to any one of claims 2 to 4, further comprising the steps of: identifying a region of the first object in the depth map, and dividing a region of the depth map that represents the region of the first object into a number of sub-regions; applying the sub-regions to the region of the first object in the 3D model; and representing the sub-regions in the images.

6. A method according to claim 5, further comprising the steps of determining whether the second object is within one of the sub-regions, or aligned with one of the sub-regions, in the second sensor data.

7. A method according to claim 5 or 6, further comprising the step of indicating a target subregion in the images.

8. A method according to claims 6 and 7, comprising the steps of: indicating a first target sub-region in the images;determining whether the second object is within the first target sub-region, or aligned with the first target sub-region in the second sensor data; andrepeating the steps of indicating and determining for each of the other sub-regions, in turn.

9. A method according to any one of claims 5 to 8, further comprising the step of recording data relating to an examination within the sub-region.

10. A method according to any preceding claim, further comprising the step of identifying a part of the second object in the second sensor data, and wherein the information indicating the position of the second object with respect to the representation of the first object comprises information indicating the position of the part of the second object with respect to the first object.

11. A method according to claim 10, wherein no other elements of the second object, aside from the part of the second object, are represented in the images.

12. A method according to any preceding claim, wherein the first sensor data is LiDAR data.

14. A method according to claim 12, further comprising the step of deriving point cloud data fromthe LiDAR data.

15. A method according to claim 14, further comprising the step of creating a 3D mesh based on the point cloud data.

16. A method according to claim 14 or 15, when dependent upon any of claims 2 to 9, comprising the step of creating the 2D depth map from the point cloud data or from the 3D mesh.

17. A method according to any one of claims 1 to 11, wherein the first sensor data is camera data.

18. A method according to any preceding claim, wherein the second sensor data is camera data.

19. A method according to any one of claims 1 to 17, wherein the second sensor data is LiDARdata.

20. A method according to any preceding claim, comprising the steps of:identifying a plurality of features of the first object;tracking the plurality of features in the second sensor data during the first time period; andmodifying the position and / or orientation of the representation of the first object during the time period based on the position and / or distance from the sensor of the plurality of features in the second sensor data.

21. A computer programme comprising instructions to carry out the method of any preceding claim, when executed on a computer.

22. A computer-readable medium having a computer programme according to claim 21 stored thereon.

23. A system comprising a computing device having a memory and one or more processors, having a computer programme according to claim 21 stored therein.

24. A system according to claim 23, further comprising a sensor device, in communication with the computing device.

25. A system according to claim 24, wherein the sensor device comprises an acoustic generator and an acoustic sensor, or an acoustic transducer.21

Images