Method and system for 3D scanning of specimens

Abstract

Systems and methods for 3D scanning of specimens are described, whereby a plurality of mounted specimens are loaded into corresponding specimen receiving locations of a multi-specimen loader, wherein each mounted specimen comprises a specimen and a specimen mount, a selected mounted specimen is transported from the multi-specimen loader to a scanning location, the selected mounted specimen is 3D scanned at the scanning location to generate a 3D representation of the selected specimen, and the selected mounted specimen is transported from the scanning location to the multi-specimen loader following 3D scanning. Other embodiments are also described and claimed.

Description

METHOD AND SYSTEM FOR 3D SCANNING OF SPECIMENSTECHNICAL FIELD

[0001] The present disclosure relates to the digitising of specimen collections. In a particular form, the present disclosure relates to forming digital 3D representations of specimens from a specimen collection.BACKGROUND

[0002] Natural history specimen collections are estimated to hold over a billion specimens comprising items ranging from mineralogical specimens to biological specimens. Clearly, these collections contain a vast amount of scientific knowledge that is essential to understanding the natural world and the changes that may be occurring in the natural environment. Unfortunately, someone who desires to examine and inspect a specimen from these collections will need to be physically present on-site to search the collection and then manipulate the specimen of interest and this can be difficult as these collections are often not easily accessible.

[0003] One type of specimen collection whose understanding is important to gain insights into local and global biodiversity are insect arthropod collections. Examples include the insect specimen collections of the National History Museum of London, Smithsonian National Museum, North Carolina State University insect museum, and the Australian National Insect Collection (ANIC). The ANIC research collection houses the world’s largest insect specimen collection (ie, currently 12 million insect specimens of 100,000 different species) and includes related groups such as mites, spiders, nematodes and centipedes.

[0004] It can be seen that the digitising of even a component of these natural history specimen collections would be desirable as it would allow researchers to remotely access, analyse and compare the specimens. Unfortunately, it is generally the case that these specimens require careful manual handling by trained operators making any scanning process time consuming and laborious. This is especially so if an attempt is made to scan specimens and create associated 3D models or representations of the specimen which will generally be of greater utility to researchers.SUMMARY

[0005] In a first aspect, the present disclosure provides a method comprising: receiving 3D specimen image data of a mounted specimen comprising a specimen and a specimen mount, the mounted specimen removably mounted to a mounting location by the specimen mount;processing the 3D specimen image data to determine an attachable portion of the mounted specimen; transferring by a transfer mechanism the mounted specimen from the mounting location to a scanning location, wherein the transfer mechanism comprises a complementary attaching arrangement that removably attaches to the attachable portion of the mounted specimen to remove the mounted specimen from the mounting location;3D scanning the mounted specimen to generate a 3D representation of the specimen; and transferring by the transfer mechanism the mounted specimen from the seaming location to the mounting location following 3D scanning.

[0006] In another form, processing the 3D specimen image data to determine an attachable portion of the mounted specimen comprises: determining from a 2D image data component of the 3D specimen image data a 2D specimen image data portion indicating where the mounted specimen is located; determining a 3D specimen image data portion corresponding to the 2D specimen image data portion where the specimen is located; and filtering the 3D specimen image data portion by a mask to determine a location of the attaching portion of the mounted specimen, wherein the mask is based on a mounting configuration of the mounted specimen.

[0007] In another form, the 3D specimen image data portion is a point cloud and filtering the 3D specimen image data portion by a mask to determine the location of the attaching portion comprises identifying one or more points in the point cloud.

[0008] In another form, the complementary attaching arrangement is a gripper and wherein filtering the 3D specimen image data portion by a mask to determine a location of the attaching portion of the mounted specimen comprises identifying a graspable region of the specimen mount by the gripper.

[0009] In another form, the graspable region is a topmost portion of the specimen mount.

[0010] In another form, the mounted specimen is selected from a number of mounted specimens each mounted to a corresponding mounting location spaced apart from each other on amounting surface, and wherein determining from a 2D image data component of the 3D specimen image data a 2D specimen image data portion indicating where the mounted specimen is located comprises initially determining an image data portion corresponding to the selected mounted specimen from image data of the entire mounting surface.

[0011] In another form, initially determining the image data portion corresponding to the selected mounted specimen from image data of the entire mounting surface comprises: determining respective image data portions corresponding to individual mounted specimens; and identifying the image data portion from the respective image data portions corresponding to the selected mounted specimen.

[0012] In another form, determining respective image data portions corresponding to individual specimens comprises processing the image data of the entire mounting surface by a classifier trained to identify individual mounted specimens.

[0013] In another form, 3D scanning the specimen comprises: orienting the mounted specimen to a plurality of different specimen orientations with respect to a camera configured to generate 2D images; capturing respective 2D camera images of the mounted specimen corresponding to the plurality of different specimen orientations; generating a 3D representation of the specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0014] In another form, orienting the mounted specimen to a plurality of different specimen orientations comprises: changing a specimen orientation; and / or changing a camera orientation.

[0015] In another form, orienting the mounted specimen to a plurality of different specimen orientations with respect to the camera comprises: fixing the camera; changing by the transfer mechanism the specimen orientation to the plurality of different specimen orientations.

[0016] In another form, generating a 3D representation of the specimen based on the respective 2D camera images and associated pose information comprises a multi-view stereo method.

[0017] In another form, generating a 3D representation of the specimen based on the respective 2D camera images and associated pose information comprises a neural radiance field based method.

[0018] In another form, the associated pose information is determined from a structure from motion analysis of the respective 2D camera images.

[0019] In another form, the associated pose information is determined from the relative positioning and orientation of the camera with respect to the specimen.

[0020] In another form, 3D scanning the specimen comprises: orienting the mounted specimen to a plurality of different specimen orientations with respect to a camera configured to generate 3D data images; capturing respective 3D data images of the mounted specimen corresponding to the plurality of different specimen orientations; generating a 3D representation of the specimen based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0021] In another form, the transfer mechanism comprises a robotic arm.

[0022] In another form, the specimen comprises an insect and the specimen mount comprises a pin extending through the insect.

[0023] In a second aspect, the present disclosure provides a 3D specimen scanning system comprising: a 3D camera for capturing 3D specimen image data of a mounted specimen comprising a specimen and a specimen mount, the mounted specimen removably mounted to a mounting location by the specimen mount; a specimen image processor comprising one or more data processors for processing the 3D specimen image data to determine an attachable portion of the mounted specimen; a transfer mechanism for transferring the mounted specimen from the mounting location to a scanning location and back to the scanning location following any scanning, wherein the transfer mechanism comprises a complementary attaching arrangement that removably attaches to the attachable portion of the mounted specimen to remove the mounted specimen from the mounting location; a 3D scanner for scanning the mounted specimen to generate a 3D representation of the specimen.

[0024] In another form, the specimen image processor is configured for: determining by the one or more processors from a 2D image data component of the 3D specimen image data a 2D specimen image data portion indicating where the mounted specimen is located; determining by the one or more processors a 3D specimen image data portion corresponding to the 2D specimen image data portion where the specimen is located; and filtering by the one or more processors the 3D specimen image data portion by a mask to determine a location of the attaching portion of the mounted specimen, wherein the mask is based on a mounting configuration of the mounted specimen.

[0025] In another form, the 3D specimen image data portion is a point cloud and filtering by the one or more processors the 3D specimen image data portion by a mask to determine the location of the attaching portion comprises identifying one or more points in the point cloud.

[0026] In another form, the complementary attaching arrangement is a gripper and wherein filtering by the one or more processes the 3D specimen image data portion by a mask to determine a location of the attaching portion of the mounted specimen comprises identifying a graspable region of the specimen mount by the gripper.

[0027] In another form, the graspable region is a topmost portion of the specimen mount.

[0028] In another form, the mounted specimen is selected from a number of mounted specimens each mounted to a corresponding mounting location spaced apart from each other on amounting surface, and wherein determining by the one or more processors from a 2D image data component of the 3D specimen image data a 2D specimen image data portion indicating where the mounted specimen is located comprises initially determining an image data portion corresponding to the selected mounted specimen from image data of the entire mounting surface.

[0029] In another form, initially determining the image data portion corresponding to the selected mounted specimen from image data of the entire mounting surface comprises: determining by the one or more processors respective image data portions corresponding to individual mounted specimens; and identifying by the one or more processors the image data portion from the respective image data portions corresponding to the selected mounted specimen.

[0030] In another form, determining by the one or more processors respective image data portions corresponding to individual specimens comprises processing the image data of the entire mounting surface by a classifier trained to identify individual mounted specimens.

[0031] In another form, the 3D scanner comprises: a camera for capturing respective 2D camera images of the mounted specimen in a plurality of different specimen orientations with respect to the 2D camera; an orientation mechanism for orienting the mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the mounted specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0032] In another form, the orientation mechanism comprises: a specimen orientation arrangement for changing a specimen orientation; and / or a camera orientation arrangement for changing a camera orientation.

[0033] In another form, the specimen orientation arrangement comprises the transfer mechanism.

[0034] In another form, the 3D representation processor is configured for generating a 3D representation of the mounted specimen based on the respective camera images and associated pose information by a multi-view stereo method.

[0035] In another form, the 3D representation processor is configured for generating a 3D representation of the mounted specimen based on the respective camera images and associated pose information by a neural radiance field based method.

[0036] In another form, the associated pose information is determined by the 3D representation processor from a structure from motion analysis of the respective 2D camera images.

[0037] In another form, the associated pose information is determined by the 3D representation processor from the relative positioning and orientation of the camera with respect to the specimen.

[0038] In another form, the 3D scanner comprises: a camera for capturing respective 3D data images of the mounted specimen in a plurality of different specimen orientations with respect to the camera; an orientation mechanism for orienting the mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the mounted specimen based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0039] In another form, the transfer mechanism comprises a robotic arm.

[0040] In another form, the specimen comprises an insect and the specimen mount comprises a pin extending through the insect.

[0041] A 3D specimen scanning system comprising means to carry out the method in accordance with the first aspect of the present disclosure.

[0042] In a third aspect, the present disclosure provides a method comprising:loading a plurality of mounted specimens into corresponding specimen receiving locations of a multi-specimen loader, wherein each mounted specimen comprises a specimen and a specimen mount; transporting a selected mounted specimen from the multi -specimen loader to a scanning location; 3D scanning the selected mounted specimen at the scanning location to generate a 3D representation of the selected specimen; and transporting the selected mounted specimen from the scanning location to the multi -specimen loader following 3D scanning.

[0043] In another form, transporting the selected mounted specimen to and from the multi-specimen loader comprises transporting the selected mounted specimen from a fixed specimen transport location corresponding to a specimen receiving location of the multi -specimen loader.

[0044] In another form, the method further comprises advancing a next mounted specimen to the fixed specimen transport location for 3D seaming.

[0045] In another form, the corresponding specimen receiving locations are arranged in a queue and advancing the next mounted specimen to the fixed specimen transport location for 3D scanning comprises moving the multi-specimen loader to advance the next mounted specimen in the queue to the specimen transport location for scanning.

[0046] In another form, the multi-specimen loader comprises a platform having a circular configuration and wherein the queue of corresponding receiving locations is arranged around a periphery of the multi - specimen loader and wherein moving the multi -specimen loader comprises rotating the multi-specimen loader.

[0047] In another form, each mounted specimen further comprises a transport mount to which the specimen mount is removably attached to and wherein the specimen receiving locations are configured to receive the transport mounts of the plurality of mounted specimens.

[0048] In another form, the transport mount comprises a magnetic material that is magnetically attracted to a complementary magnetic material located at the specimen receiving location and / or scanning location to positively position the mounted specimen at the specimen receiving location and / or scanning location.

[0049] In another form, transporting the selected mounted specimen to and from the multi-specimen loader comprises: releasably attaching a transport mechanism to a transport mount of the selected mounted specimen;transporting by the transport mechanism the selected mounted specimen to the scanning location for 3D scanning; transporting the selected mounted specimen back from the scanning location to the multispecimen loader following 3D scanning; and detaching the transport mechanism from the transport mount of the selected mounted specimen.

[0050] In another form, the transport mechanism comprises a loading arm configured to releasably attach to the transport mount when moving in a first loading direction with respect to the transport mount and to detach from the transport mount when moving in an unloading direction with respect to the transport mount.

[0051] In another form, the loading arm is rotatable to releasably attach to the transport mount when rotating in a loading direction and to detach when rotating in an unloading direction.

[0052] In another form, 3D scanning the selected specimen comprises: orienting the selected mounted specimen to a plurality of different specimen orientations with respect to a camera configured to generate 2D images; capturing respective 2D camera images of the selected mounted specimen corresponding to the plurality of different specimen orientations; generating a 3D representation of the specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0053] In another form, orienting the selected mounted specimen to a plurality of different specimen orientations comprises: changing a specimen orientation; and / or changing a camera orientation.

[0054] In another form, changing a specimen orientation comprises rotating the selected mounted specimen about a vertical axis substantially extending through the mounted specimen.

[0055] In another form, changing a camera orientation comprises rotating the selected mounted specimen about a horizontal axis substantially extending through the mounted specimen.

[0056] In another form, generating a 3D representation of the specimen based on the respective camera images and associated pose information comprises a multi-view stereo method.

[0057] In another form, generating a 3D representation of the specimen based on the respective camera images and associated pose information comprises a neural radiance field based method.

[0058] In another form, the associated pose information is determined from a structure from motion analysis of the respective 2D camera images.

[0059] In another form, the associated pose information is determined from the relative positioning and orientation of the camera with respect to the specimen.

[0060] In another form, 3D scanning the selected specimen comprises: orienting the selected mounted specimen to a plurality of different specimen orientations with respect to a camera configured to generate 3D data images; capturing respective 3D data images of the selected mounted specimen corresponding to the plurality of different specimen orientations; generating a 3D representation of the specimen based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0061] In another form, the specimen comprises an insect and the specimen mount comprises a pin extending through the insect.

[0062] In a fourth aspect, the present disclosure provides a 3D specimen scanning system comprising: a multi-specimen loader for loading a plurality of mounted specimens for scanning, wherein the plurality of mounted specimen are loaded into corresponding specimen receiving locations of the multispecimen loader, wherein each mounted specimen comprises a specimen and a specimen mount; a transport mechanism for transporting a selected mounted specimen from the multi-specimen loader to a scanning location and back to the multi-specimen loader following any 3D scanning; and a 3D scanner for 3D scanning the selected mounted specimen and generating a 3D representation of the selected specimen.

[0063] In another form, the transport mechanism is configured to transport the selected mounted specimen from a fixed specimen transport location corresponding to a specimen receiving location of the multi-specimen loader.

[0064] In another form, the multi-specimen loader is configured to advance a next mounted specimen to the fixed specimen transport location for 3D scanning.

[0065] In another form, the corresponding specimen receiving locations are arranged in a queue and the multi-specimen loader moves to advance the next mounted specimen in the queue to the specimen transport location for scanning.

[0066] In another form, the multi-specimen loader comprises a platform having a circular configuration and wherein the queue of corresponding receiving locations is arranged around a periphery of the multispecimen loader and wherein moving the multi -specimen loader comprises rotating the multi-specimen loader.

[0067] In another form, each mounted specimen further comprises a transport mount to which the specimen mount is removably attached to and wherein the specimen receiving locations are configured to receive the transport mounts of the plurality of mounted specimens.

[0068] In another form, the transport mount comprises a magnetic material that is magnetically attracted to a complementary magnetic material located at the specimen receiving location and / or scanning location to positively position the mounted specimen at the specimen receiving location and / or scanning location.

[0069] In another form, the transport mechanism is configured to: releasably attach to the transport mount of the selected mounted specimen; transport the selected mounted specimen to the scanning location for 3D scanning; transport the selected mounted specimen back from the scanning location to the multi-specimen loader following 3D scanning; and detach from the transport mount of the selected mounted specimen.

[0070] In another form, the transport mechanism comprises a loading arm configured to releasably attach to the transport mount when moving in a first loading direction with respect to the transport mount and to detach from the transport mount when moving in an unloading direction with respect to the transport mount.

[0071] In another form, the loading arm is rotatable to releasably attach to the transport mount when rotating in the loading direction and to detach when rotating in the unloading direction.

[0072] In another form, the 3D scanner comprises: a camera for capturing respective 2D camera images of the selected mounted specimen in a plurality of different specimen orientations with respect to the camera; an orientation mechanism for orienting the selected mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the selected specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0073] In another form, the orientation mechanism comprises:a specimen orientation arrangement for changing a specimen orientation; and / or a camera orientation arrangement for changing a camera orientation.

[0074] In another form, the specimen orientation arrangement is configured to rotate the selected mounted specimen about a vertical axis substantially extending through the mounted specimen.

[0075] In another form, the camera orientation arrangement is configured to rotating the selected mounted specimen about a horizontal axis substantially extending through the mounted specimen.

[0076] In another form, the 3D representation processor is configured for generating a 3D representation of the specimen based on the respective camera images and associated pose information by a multi -view stereo method.

[0077] In another form, the 3D representation processor is configured for generating a 3D representation of the specimen based on the respective camera images and associated pose information by a neural radiance field based method.

[0078] In another form, the associated pose information is determined by the 3D representation processor from a structure from motion analysis of the respective 2D camera images.

[0079] In another form, the associated pose information is determined is determined by the 3D representation processor from the relative positioning and orientation of the camera with respect to the specimen.

[0080] In another form, the 3D scanner comprises: a camera for capturing respective 3D data images of the selected mounted specimen in a plurality of different specimen orientations with respect to the camera; an orientation mechanism for orienting the selected mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the selected specimen based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0081] In another form, the specimen comprises an insect and the specimen mount comprises a pin extending through the insect.

[0082] A 3D specimen scanning system comprising means to carry out the method in accordance with the third aspect of the present disclosure.BRIEF DESCRIPTION OF DRAWINGS

[0083] Embodiments of the present disclosure will be discussed with reference to the accompanying drawings wherein:

[0084] FIG. 1 is a flow diagram of an example method for generating a 3D representation of a specimen in accordance with some embodiments;

[0085] FIG. 2A is a system overview diagram of an example 3D specimen scanning system for generating a 3D representation of a specimen in accordance with some embodiments;

[0086] FIG. 2B is a system overview diagram of an example 3D scanner in accordance with some embodiments;

[0087] FIGS. 3A and 3B are top and end-on sectional figurative views of a specimen and associated specimen mounting arrangement in accordance with some embodiments;

[0088] FIG. 4 is a flow diagram of an example method for visually processing specimen image data to determine an attachable portion in accordance with some embodiments;

[0089] FIG. 5 is a top view image of a number of specimens each mounted to a corresponding mounting location spaced apart from each other on a mounting surface in accordance with some embodiments;

[0090] FIG. 6 is a top view image of a number of specimens following processing by a classifier trained to identify individual specimens in an image showing the determined image data portion for each specimen;

[0091] FIG. 7 is atop view image of the specimens illustrated in FIG. 6 further showing the location of the identified attachable portions 710A-I of the mounted specimens in accordance with some embodiments;

[0092] FIG. 8 is a flow diagram of an example method for 3D scanning the specimen in accordance with some embodiments;

[0093] FIGS. 9A-9D are figurative views showing the operation of an example 3D specimen scanning system in accordance with some embodiments;

[0094] FIG. 10 shows a selection of images of specimens and the associated generated 3D representation of the specimens in accordance with some embodiments;

[0095] FIG. 11 is a flow diagram of an example method for generating a 3D representation of a specimen in accordance with some embodiments;

[0096] FIG. 12A is a system overview diagram of an example 3D specimen scanning system for generating a 3D representation of a specimen in accordance with some embodiments;

[0097] FIG. 12B is a system overview diagram of an example 3D scanner in accordance with some embodiments;

[0098] FIG. 13A is a figurative diagram of an example multi-specimen loader in accordance with some embodiments;

[0099] FIG. 13B is figurative diagram of another example multi -specimen loader in accordance with some embodiments;

[0100] FIG. 14A is a flow diagram of an example method for transporting a mounted specimen to and from a multi-specimen loader in accordance with some embodiments;

[0101] FIG. 14B is a flow diagram of an example method for 3D scanning the mounted specimen in accordance with some embodiments;

[0102] FIG. 15A and 15B are front and rear perspective views of a 3D specimen scanning system in accordance with some embodiments;

[0103] FIG. 16 is an end exploded view of the 3D specimen scanning system illustrated in FIGS. 15A and 15B;

[0104] FIG. 17 is a perspective view of a mounted specimen in accordance with some embodiments;

[0105] FIG. 18A is a perspective view of a multi -station loader in accordance with some embodiments;

[0106] FIG. 18B is a top view of the multi-station loader illustrated in FIG. 18 A;

[0107] FIG. 19A is a perspective view of a transport mechanism in accordance with some embodiments;

[0108] FIG. 19B is a top view of the transport mechanism illustrated in FIG. 19A;

[0109] FIG. 20 is an exploded perspective view of a 3D scanner in accordance with some embodiments;

[0110] FIG. 21 is a plot of estimated camera poses captured by a 2D camera showing the effect of varying azimuth and elevation angle in accordance with some embodiments; and

[0111] FIG. 22 shows images of five specimens (ROW A) and the associated 3D representations (ROWS B, C and D) generated in accordance with some embodiments.

[0112] In the following description, like reference characters designate like or corresponding parts throughout the figures.DESCRIPTION OF EMBODIMENTS

[0113] Techniques are disclosed for automating the generation of 3D representations of specimens, such as insect specimens, where the specimens typically require a large degree of manual handling due to a combination of their fragile nature and their mounting arrangements. The disclosed techniques are particularly beneficial, as they allow for the processing of large numbers of specimens and the subsequent generation of high quality 3D representations that enable the remote study of these specimens.

[0114] Referring now to FIG. 1, there is shown a flow diagram of a method 100 for generating a 3D representation of a mounted specimen according to an illustrative embodiment. By way of overview, method 100 comprises (at block 110) receiving 3D specimen image data of a mounted specimen (comprising a specimen and associated specimen mount) removably mounted to a mounting location.

[0115] In one example, as will be seen below, the specimen may comprise an insect mounted to a mounting location such as a specimen box incorporating a mounting surface such as foam or the like by a specimen mount comprising a pin that extends through the specimen. In this example, the insect specimen is located approximately mid-way down the pin and the pin is then removably inserted into the mounting surface. This type of specimen mount is common for mounting delicate and fragile biological specimens of a certain size and shape as it allows for the specimen to be stored above and remote from the mounting surface and inserted or removed by manipulation of the pin without having to otherwise touch the specimen.

[0116] In one example, the 3D specimen image data comprises a 2D image data component (eg, an RGB image) and 3D image data component (eg, depth map data) of the mounted specimen. At block 120, the 3D specimen image data is visually processed to determine an attachable portion of the mountedspecimen. The mounted specimen is then transferred (at block 130) by a transfer mechanism from the mounting location to a scanning location, where the transfer mechanism comprises a complementary attaching arrangement or means that removably attaches to the attachable portion of the mounting arrangement to remove the mounted specimen from the mounting location. Following transfer, the specimen is subject to 3D scanning (at block 140) to generate a 3D representation of the specimen. In one example, the 3D scanning comprises the specimen being presented in multiple orientations with respect to a scanning camera by the transfer mechanism as part of the 3D scanning process. Finally, the specimen is transferred (at block 140) from the scanning location to the mounting location by the transfer mechanism following 3D scanning.

[0117] Referring now to FIG. 2A, there is shown an example 3D specimen scanning system 200 for generating a 3D representation of a specimen according to an illustrative embodiment. In various examples, system 200 may be configured to implement or carry out method 100 as illustrated in FIG. 1 (and also FIGS. 4 and 8 referred to below).

[0118] In this example, 3D specimen scanning system 200 comprises a 3D camera 210 for capturing 3D specimen image data of a mounted specimen and a specimen image processor 220 comprising one or more data processors 221 (and associated data storage 222) for receiving and visually processing the 3D specimen image data to determine an attachable portion of the mounted specimen to allow transfer of the mounted specimen. 3D specimen scanning system 200 further comprises a transfer mechanism 230 for transferring the mounted specimen from the mounting location to a scanning location and back to the mounting location following any scanning. The transfer mechanism 230 also comprises a complementary attaching arrangement that removably attaches to the attachable portion of the mounted specimen to remove the mounted specimen from the mounting location. 3D specimen scanning system further comprise a 3D scanner 240 that scans the mounted specimen to generate a 3D representation of the specimen.

[0119] As referred to above, image processor includes one or more data processors 221 configured to configured to receive and process the 3D specimen image data of the specimen and process this specimen to determine an attachable portion of the mounted specimen. Image processor 220 may be any computing device or system computer system, such as a workstation, desktop computer, server, laptop, handheld computer, tablet computer, mobile computing or communication device, VR device or VR component (eg, headset, camera, etc) or other form of computing or telecommunications device that is capable of communication and that has sufficient processor power and memory capacity to perform the operations described in this disclosure.

[0120] In this example, image processor 220 includes storage 222 that may include one or more storage devices or non-transitory computer-readable media having encoded on the media one or morecomputer-executable instructions or software for implementing techniques as variously described in this disclosure. The storage 222 may include a computer system memory or random access memory, such as a durable disk storage (which may include any suitable optical or magnetic durable storage device, eg, RAM, ROM, Flash, USB drive, or other semiconductor-based storage medium), a hard-drive, CD-ROM, or other computer readable media, for storing data and computer-readable instructions or software that implement various embodiments as taught in this disclosure.

[0121] Storage 222 may include other types of memory as well, or combinations. The storage device may be provided on image processor 220 or provided separately or remotely. The non-transitory computer-readable media may include, but are not limited to, one or more types of hardware memory, non-transitory tangible media (for example, one or more magnetic storage disks, one or more optical disks, one or more USB flash drives), and the like. The non transitory computer-readable media included in the image processor 220 may store computer-readable and computer-executable instructions or software for implementing various embodiments. The computer-readable media may be provided on the image processor 220 or provided separately or remotely from the image processor 220.

[0122] The one or more processors 220 are configured for executing computer-readable and computer-executable instructions or software stored in the storage 221 and other programs for controlling system hardware. Virtualization may be employed in the image processor 220 so that infrastructure and resources may be shared dynamically. For example, a virtual machine may be provided to handle a process running on multiple processors so that the process appears to be using only one computing resource rather than multiple computing resources. Multiple virtual machines may also be used with one processor. In other examples, a distributed image processing system may be provided including a plurality of such image processors 220.

[0123] A user may interact with the specimen image processor 220 through a display 224, such as a screen or monitor, including an augmented reality display device, which may display one or more user interfaces provided in accordance with some embodiments. The output device 224 may also display other aspects, elements or information or data associated with some embodiments. Image processor 220 may include input or input / output devices 223 for receiving input from a user, for example, a keyboard, a joystick, a game controller, a pointing device (eg, a mouse, a user's finger interfacing directly with a touch-sensitive display device, etc.), or any suitable user interface, including an AR headset. The image processor 220 may include other suitable conventional I / O peripherals.

[0124] Specimen image processor 220 may run any suitable operating system including, but not limited to, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating systems for mobile computing devices, or any other operating system capable of running on image processor and performing the operations described inthis disclosure. In an embodiment, the operating system may be run on one or more cloud machine instances.

[0125] In other embodiments, the functional components / modules of specimen image processor 220 may be implemented with hardware, such as gate level logic (eg, FPGA) or a purpose-built semiconductor (eg, ASIC). Still other embodiments may be implemented with a microcontroller having several input / output ports for receiving and outputting data, and several embedded routines for carrying out the functionality described in this disclosure. In a more general sense, any suitable combination of hardware, software, and firmware can be used, as will be apparent.

[0126] As will be appreciated in light of this disclosure, the functionality of specimen image processor 220 may be implemented in software, such as a set of instructions (eg, HTML, XML, C, C++, object oriented C, BASIC, Python, etc.) encoded on any computer readable medium or computer program product (eg, hard drive, server, disc, or other suitable non-transitory memory or set of memories), that when executed by one or more processors, cause the various methodologies provided in this disclosure to be carried out.

[0127] Referring now to FIG. 2B, there is shown a system overview diagram of a 3D scanner which in one example may comprise 3D scanner 240 illustrated in FIG. 2A. In this example, 3D scanner 240 comprises a camera 241 for capturing respective 2D camera images of the mounted specimen in a plurality of different specimen orientations with respect to the camera and an orientation mechanism 242 for orienting the mounted specimen to the plurality of different specimen orientations. 3D scanner 240 further comprises a 3D representation processor 243 comprising one or more data processors for generating the 3D representation of the mounted specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0128] In various embodiments, 3D representation processor 243 may have equivalent processing functionality to specimen image processor 220 (including storage, display and input device) as described above except configured to generate the 3D representation in accordance with the present disclosure. In other embodiments, 3D representation processor 243 and image processor 220 may comprise the same computing system or device.

[0129] Referring now to FIGS. 3A and 3B, there are shown top and end-on sectional figurative views 300, 350 of an example mounted specimen comprising a specimen 310 and specimen mount 330 which may be processed in accordance with embodiments of the present disclosure. In this example, the specimen comprises an insect 310 that is mounted to mounting location 320 in the form of specimen box by a mounting arrangement in the form of a pin 330 having a head region 335 (ie, a pin head) with a diameter of approximately 1.2 mm.

[0130] Referring back to FIG. 1, at block 110 3D specimen image data is received of the mounted specimen mounted to the mounting location., In one example, the 3D specimen image data comprises a 2D image data component and a 3D image data component corresponding to the mounted specimen taken by a camera located above the specimen such as the view depicted in FIG. 3A.

[0131] In one example, the 2D image data component of the 3D specimen image data comprises a registered RGB image Irgband the 3D image data component comprises a depth map I^. In one example, the camera is an RGB-D sensor or camera having a sub-mm depth resolution. In one example, the RGB-D sensor is a Zivid Two M70 camera having a 2D resolution of 1944 x 1200 pixels (2.3 MP) and generating depth map data in the form of a 3D point cloud with spatial resolution of approximately 0.39 mm at a working distance of 700 mm.

[0132] At block 120 of FIG. 1, the specimen image data is processed to determine an to determine an attachable portion of the mounted specimen. Referring now to FIG. 4 there is shown a is a flow diagram of an example method 400 for visually processing specimen image data to determine an attachable portion in accordance with some embodiments.

[0133] Initially, method 400 at block 410 comprises determining from a 2D image data component of the 3D specimen image data a 2D specimen image data portion indicating where the mounted specimen is located. In one example, the 2D specimen image data portion corresponds to a bounding box having values Nbbconsisting of the top left and bottom right pixel positions in the 2D image data component (eg, Irgb) of 3D specimen image data that bound the specimen in the 2D image data.

[0134] At block 420, the 3D specimen image data portion corresponding to the 2D specimen image data portion where the specimen is located is determined. In one example, the determined 3D specimen image data portion will be in the form of a point cloud X corresponding to the mounted specimen for the region corresponding to bounding box Nbb.

[0135] At block 430, the 3D specimen image data portion is filtered by a mask to determine a location of the attaching portion of the mounted specimen where the mask is based on the mounting configuration of the mounted specimen.

[0136] Consider the mounting arrangement configuration shown in FIG. 3, where in this example the mounting configuration comprises a pin known to have a diameter of 1 mm and a height of 35 mm. Based on this mounting configuration, a mask may be defined based on the geometry of the pin to select for an attachable region corresponding to the upper or head region of the pin from the point cloud X and the the location of the attachable portion Pxyzmay then be determined. In the exampledescribed with reference to FIG. 3, the attachable portion Pxyzwill correspond to the topmost portion or location of the fdtered selected depth data corresponding to the pin head 335 obtained from the application of the mask.

[0137] As would be appreciated, other types of masks may be determined based on the mounting configuration. As an example, the attachable region may be located to the sides or at a peripheral region of the mounted specimen in which case the mask can be configured to identify the relevant region from the point cloud.

[0138] While in the above example, the attachable portion Pxyzfor a mounted specimen is determined from specimen image data corresponding to a top view (ie, directly above), in other examples the attachable portion Pxyzmay be determined from specimen image data taken from other directions not necessarily viewed from directly above which may be more applicable depending on the mounting arrangement configuration. In yet another example, the attachable portion Pxyzmay be determined based on specimen image data taken corresponding to images taken from multiple directions.

[0139] As would be appreciated, in different examples the specimen that is to be 3D scanned may be arranged with multiple other specimens that are mounted nearby.

[0140] Referring now to FIG. 5, and by analogy with FIG. 3, there is shown a top view image 500 of a number of mounted specimens 510A-G each mounted to a corresponding mounting location spaced apart from each other on a mounting surface 550 of a specimen box 570. As would be appreciated, it is often convenient to group multiple related specimens together in respective specimen boxes where they can be then stored in individual specimen drawers. Accordingly, in this example, the 3D specimen image data that is captured and received may correspond to multiple specimens spaced over a mounting surface such as shown in FIG. 5 and determining the 2D image data component in which the specimen of interest is located comprises initially determining the image data portion corresponding to the specimen from image data of the entire mounting surface 550.

[0141] In one example, initially determining the image data portion corresponding to the specimen from image data of the entire mounting surface comprises determining the respective image data portions from the image of the mounting surface that correspond to individual mounted specimens and then identifying the image relevant image data portion from the respective image data portions that corresponds to the specimen of interest. In one example, determining respective image data portions corresponding to individual specimens comprises processing the image data of the entire mounting surface by a classifier trained to identify individual specimens in an image.

[0142] In one example, a pre-trained deep learning based object detection model is employed to determine a bounding box in the 2D image corresponding to each specimen. In another example, the object detection model (eg, YOLOv7) has been fine-tuned based on labelled data comprising examples of the mounted specimens of interest. In other examples, object detection models such as Mask R-CNN or YOLOv7 may be employed.

[0143] In one example, the fine tuning process involves initial pre-training on an image data set (eg, ImageNet and / or COCO datasets) and then fine tuning based on an extended dataset comprising labelled cropped specimen images corresponding to the specimen type of interest (eg, insect specimens). In one example, the extended dataset is divided into training / validation / test and the object detection model is trained on the training data set and tuned using the validation data set.

[0144] Referring now to FIG. 6, there is shown atop view image 600 of a number of specimens 610A-I following processing by a classifier trained to identify individual specimens in an image showing the determined image data portion 615A-I for each specimen. In this example, image data portions are in the form of bounding boxes for each specimen. In this manner, a list of N segmented speciments and their bounding box values Nbbwhich consists of the top left and bottom right pixel positions may be identified.

[0145] Referring now to FIG. 7, there is shown a top view image 700 of the specimens illustrated in FIG. 6 further showing the location of the identified attachable portions 710A-I (ie, individual locations P^yz for the ithspecimen) following the process indicated in FIG. 4 corresponding to the respective pin heads of each of the mounting pins.

[0146] Referring back to FIG. 1, at block 130 the specimen is transferred by a transfer mechanism to a scanning location. In various embodiments, the transfer mechanism comprises a complementary attaching arrangement that removably attaches to the attachable portion of the mounted specimen to remove the mounted specimen from the mounting location. As would be appreciated, transfer mechanism may comprise any electronically controlled mechanism that includes an attaching arrangement that is adapted to removably attach to the attachable portion of the mounting arrangement.

[0147] In one example, the transfer mechanism comprises a robotic arm comprising at least two or more arm members and associated rotating joints that is controllable to position an operative end comprising an attaching arrangement to a 3D location. In one example, the robotic arm comprises multiple joints (eg, base, shoulder and / or elbow) and a rotatable operative end (eg, rotatable in 1, 2 or 3 rotation axes) where the attaching arrangement is in the form of a gripper (eg, two finger, three finger or bio gripper) that may be controlled to grip an attachable portion or region of the mounting arrangement in the form of a graspable region. In one example, considering the mounted specimen illustrated in FIGS. 3A and 3B, the gripper may function to grip or grasp the pin head (ie, the graspable region) that has beenidentified in accordance with the present disclosure and remove the mounted specimen from the mounting location to be transferred to the scanning location.

[0148] In another example, the attaching arrangement may comprise an electromagnetic arrangement that may be activated to removably attach to an attachable portion of the mounting arrangement that is formed of a magnetic material. In other examples, the attaching arrangement may comprise an interlocking member that is able to removably interlock with an interlocking portion forming part of the mounting arrangement to allow transfer of the specimen to the scanning location.

[0149] In another example, the robotic arm may have telescoping elements that may be controlled to change the length of one or more of the arm members comprising the robotic arm. In another example, the transfer mechanism may comprise a gantry arrangement where the attachment arrangement, following attachment to the attachment portion of the mounting arrangement of the specimen, moves along a rail or frame to transfer the specimen to a scanning location.

[0150] In another example, the transfer mechanism may comprise a transport mechanism such as a loading arm (eg, see FIGS. 19A and 19B).

[0151] Referring back to FIG. 1, at block 140 the specimen is 3D scanned to generate a 3D representation of the specimen. In one example, the specimen is presented in multiple orientations by the transfer mechanism.

[0152] Referring now to FIG. 8, there is shown an example method 800 for 3D scanning the specimen according to some embodiments. At block 810, method 800 comprises orienting the mounted specimen to a plurality of different specimen orientations by an orientation mechanism with respect to a camera configured to generate 2D images. As would be appreciated, 3D cameras may be generally configurable to generate 2D images. In one example, the orientation mechanism comprises a specimen orientation arrangement for changing or modifying a specimen orientation and a camera orientation arrangement for changing or modifying the camera orientation.

[0153] In one example, the specimen orientation arrangement comprises the transfer mechanism which is configured to adopt the two or more different specimen orientations with respect to the camera while the location and orientation of the camera is fixed. In another example, the mounted specimen remains stationary, and the camera is reoriented by the camera orientation arrangement with respect to the stationary specimen to generate the two or more different orientations with respect to the camera. In yet another example, both the orientation of the specimen and the camera may be changed relative to each other in order to achieve the different specimen orientations with respect to the camera.

[0154] At block 820, respective 2D camera images of the mounted specimen are captured corresponding to the plurality of different specimen orientations.

[0155] At block 830, 3D representation of the specimen is generated based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0156] In one example, a multi-view stereo method is used to generate the 3D representation from the multiple 2D images and associated pose information (eg, multi-view stereo pipeline of COLMAP). In another example, a neural radiance field (NeRF) based method is adopted to generate the 3D representation from the multiple 2D images and associated pose information (eg, nerfstudio).

[0157] Other example methods for generating a 3D representation that may be used in accordance with the present disclosure include, but are not limited to, NeRF, Mip-NERF, Neus, Nerfacto, Nerfacc, gaussian splatting or Instant-NG.

[0158] In one example, the pose information is determined from the known relative positioning and orientation of the camera with respect to the specimen as determined by the specimen and camera orientation arrangements. In one example, the motor controller values used to control the specimen and camera orientation arrangements to their desired orientations may be employed to derive the camera pose with respect to the mounted specimen. In other examples, the orientations of the specimen and camera orientation arrangements may be directly measured, such as by the use of angular encoders, and these measured orientations may be employed to derive the camera pose with respect to the mounted specimen. In the example where the specimen orientation arrangement comprises the transfer mechanism (eg, robotic arm) the camera pose may be determined by the transfer mechanism controller values and / or measured orientation of the transfer mechanism to derive the camera pose with respect to the mounted specimen.

[0159] In another example, the pose information is determined from the camera image. In one example, the pose information is determined from the camera image using a structure -from -motion method (eg, structure -from-motion pipeline from COLMAP or Alicevision Meshroom). In another example, the pose information is determined from both the relative positioning and orientation of the camera (as referred to above) and any estimate determined from the camera image (such as from features detected).

[0160] In one example, the 3D representation is in the form of a 3D mesh. In another example, the 3D mesh is generated by Poisson surface reconstruction method that functions to convert a pointcloud adopting a smoothness prior. In another example, a Neural Kernel Field approach may be adopted that uses a data driven prior along with the smoothness prior.

[0161] In another example, 3D scanning the specimen comprises orienting the mounted specimen to a plurality of different specimen orientations with respect to a camera or sensor configured to generate 3D data images. In one example, the camera may be an RGBD camera that also generates 3D depth data. In another example, the camera may comprise a laser scanner (eg, LiDAR and / or line profile scanner) that generates a 3D data image in the form of a point cloud. Respective 3D data images of the mounted specimen corresponding to the plurality of different specimen orientations are then captured and a 3D representation of the specimen may be generated based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0162] In one example, where the camera comprises a RGBD camera, a visual odometry based method or approach is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, KinectFusion). In another example, a simultaneous localization and mapping (SLAM) based method is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, ORB-SLAM2, BAD-SLAM, or Gradient-SDF).

[0163] In another example, where the camera comprises a laser scanner, a point cloud registration from motion control approach (eg, see Wang., 2021, “3D Reconstruction Using a Linear Laser Scanner and a Camera'”, in 2021 2nd International Conference on Artificial Intelligence and Computer Engineering (ICAICE), pp. 666-673, IEEE) is used to generate the 3D representation from the multiple 3D data images and associated pose information. In another example, a SLAM based method is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, Visual SLAM or WildCAT).

[0164] In various examples, the 3D representation generated from the multiple 3D data images may be in the form of including, but not limited to, a 3D point cloud, a 3D mesh, a Signed Distance Field (SDF) that may in turn be interrogated to extract a 3D mesh, or a deep network encoding the 3D representation.

[0165] Referring back to FIG. 1, following 3D scanning, at block 150 the mounted specimen is transferred from the scanning location to the mounting location by the transfer mechanism. In the example, where there are multiple specimens, the next specimen may then be selected for processing in accordance with the present disclosure.

[0166] Referring now to FIGS. 9A-9D, there are shown figurative views showing the operation of an example 3D specimen scanning system 900. As would be appreciated, 3D specimen scanningsystem 900 is just one example embodiment of the general 3D specimen scanning system 200 illustrated in FIG. 2 above.

[0167] In this example, 3D specimen scanning system comprises a 3D camera 910 for capturing 3D specimen image data of a mounted specimen 310 mounted to a mounting location 320 by specimen mount 330 (see also FIG. 3B) and a specimen image processor 920 comprising one or more data processors for visually processing the 3D specimen image data as will be described below.

[0168] Scanning system 900 further comprises a transfer mechanism 930 for transferring the mounted specimen 310 from the mounting location 320 to a scanning location 940. In this example, transfer mechanism 930 comprises a robotic arm having a first arm member 932 rotationally mounted about a horizontal axis to a base member 931 by a “shoulder” joint. In this example, base member 931 comprises an upper portion 93 la to which the first arm member is mounted to and a lower static base portion 93 lb where upper portion 93 la is rotatably mounted to based portion 93 lb to allow rotation about a vertical axis.

[0169] Attached to first arm member 932 by an “elbow” joint is second arm member 933 where the elbow joint allows the second arm member 932 to rotate with respect to the first arm member 931 about a horizontal axis. Robotic arm further comprises a hand member 934 that is configured so that an operative end 935 of the hand member 934 is rotatable about three orthogonal axis about three “wrist” joints providing in total 6 degrees of freedom for robotic arm 930. As would be appreciated, robotic arm 930 incorporates associated motors and control software allowing the position of the operative end to be controlled to move to a selected position upon command. In this example, operative end 935 of hand member 934 comprises a robotic gripper 937 that may be commanded to grip with a specified force and release.

[0170] In this example, 3D camera 910 for capturing 3D specimen image data is mounted to hand member 934 and as such forms an “eye-in-hand” camera where the relationship between the camera reference coordinate system and the robotic arm reference coordinate system, typically defining the coordinates (eg, cartesian) of the end effector of the robotic arm, may be determined by a calibration procedure to allow mapping or transforming of the measurements in camera co-ordinates to measurements in the robot arm coordinate frame.

[0171] Scanning system 900 also includes a 3D scanner 950 for 3D scanning the specimen in multiple orientations to generate a 3D representation of the specimen 310. In this example, 3D scanner 950 includes a high resolution 2D camera 953 and camera mount 954 that positions the camera 953 at an elevated position to allow viewing of the specimen. In one example 2D camera 953 comprises a Basler acA4096-30uc CMOS based sensor with a resolution of 8.9 MP resolution combined with 35 mm focal-length lens configured to capture an image of the specimen at a distance of approximately 25cm. As would be appreciated lens of other focal lengths may be used depending on the size of the specimen.

[0172] As shown in FIG. 9A, the camera 910 mounted to the hand member 935 is positioned above the mounting location 320 and captures 3D specimen image data comprising both 2D and 3D image data components of the specimen 310 and specimen mount 330. The 3D specimen image data is then processed to determine an attachable portion of the mounting specimen by specimen image processor 920 as has been previously described. In this example, the specimen mount comprises a pin 330 that extends through the specimen 310 and the attachable portion is the pin head 335.

[0173] At FIG. 9B, the robotic arm is controlled to grip the pin head using gripper 937 and remove the mounted specimen from the mounting location and transfer the specimen to the scanning location 940 as shown in FIG. 9C. In one example, the pinhead or attachable portion Pxyzas determined by specimen image processor 920 in the camera reference coordinate system of camera 910 is transformed to the robotic arm coordinate system to give Qxyz. In one example, the transformation is carried out adopting coordinate transformation matrix H^am which is determined during calibration of camera 910.

[0174] The transformed Qxyzcorresponding to the attachable portion 335 is then provided to the robotic arm 930 which in this example is controlled to be initially locate 200 mm above the pinhead 335 and then to move down vertically to pick up and transfer the specimen 310 using the gripper 937.

[0175] 3D scanning of the specimen as shown in FIG. 9C comprises in this example capturing multiple 2D images of the specimen 310 in different orientations in order to generate a 3D representation from the multiple 2D images. In this example, specimen 310 is oriented by the transfer mechanism (ie, robot arm 930) to adopt different orientations by rotating the specimen 310 about a vertical axis A (substantially corresponding to the pin axis) in front of camera 953 to capture the multiple 2D images corresponding to the different orientations. In one example, the specimen is rotated 360 degrees in 5 degree increments to provide 72 images of the specimen from which the 3D representation is generated.

[0176] As would be appreciated, the different orientations need not be limited to rotation about a vertical axis and in other examples the elevation or tilt angle of the specimen 310 may also be varied to adopt different orientations. In yet other examples, the specimen may be rotated about a tilted or elevated axes. In other embodiments, camera 953 may be configured to move while the specimen 310 is held in a fixed orientation and location. In one example, the camera 953 may be mounted to a rail or track or other mounting member that moves and / or changes the orientation of the camera 953 with respect to the specimen 310. In another example, camera 953 itself may be mounted to a robotic arm and controlled toadopt different orientations with respect to mounted specimen 310. In yet another example, the specimen is rotated 360 degrees in fixed increments (eg, 5 degree) for a given camera position, and the camera is then rotated about the specimen and specimen again rotated through 360 degrees. This process may then be repeated for different camera locations with respect to the specimen to further increase the number of captured images. In another example, the rotation angles (in any axis) are selected to maintain a uniform pose density with respect to the specimen. In one example, the rotation angles are selected to maintain the geodesic distance between the camera and the specimen.

[0177] In another example, the different orientations may be selected based on the geometry or configuration of the specimen to be scanned. As an example, the different orientations adopted for the situation where the camera is viewing a region of the specimen having more variation in the surface geometry may be selected to be more closely spaced or denser in contrast to where the camera is viewing a region of the specimen where the surface geometry does not vary considerably. In one example, the different orientations may be selected to follow the surface profile to maintain a substantially constant distance between the camera and the region of the specimen being viewed.

[0178] In another example, 3D scanning of the specimen as shown in FIG. 9C comprises capturing multiple 3D images of the specimen 310 in different orientations by a camera or sensor configured to generate 3D data images such as an RGBD camera that also generates 3D depth data or a laser scanner (eg, LiDAR and / or line profile scanner) that generates a 3D data image in the form of a point cloud.

[0179] Following the 3D scanning, the specimen is transferred from the scanning location 950 to the mounting location 320 as shown in FIG. 9D and remounted.

[0180] Referring now to FIG. 10, there is shown a selection of images of specimens (ROW A) and the associated 3D representations based on 3D mesh reconstructions from a stereo-view model (ROW B) and 3D mesh reconstructions from neural radiance field model (ROW C) generated in accordance with the present disclosure. As can be seen from inspection, the generated 3D representations comprise a high level of detail and include minute features of the specimen such as insect antennae.

[0181] Referring now to FIG. 11, there is shown a flow diagram of a method 1100 for generating a 3D representation of a selected specimen according to an illustrative embodiment. By way of overview, method 1100 comprises (at block 1110) loading a plurality of mounted specimens into corresponding specimen receiving locations of a multi -specimen loader where mounted specimen comprises the specimen and the associated specimen mount. At block 1120, the selected mounted specimen is transported from the multi -specimen loader to a scanning location. At block 1130, 3D the selected mounted specimen is 3D scanned at the scanning location to generate a 3D representation of the selectedspecimen. Following 3D scanning the mounted specimen is then transported back to the multi-specimen loader (at block 1140).

[0182] In one example, transporting the selected mounted specimen to and from the multispecimen loader comprises transferring the selected mounted specimen from a fixed specimen transport location corresponding to a specimen receiving location of the multi-specimen loader. In another example, following 3D scanning of the selected mounted specimen, the multi-specimen loader is moved to advance a new mounted specimen to the fixed specimen transport location fortransporting and 3D scanning.

[0183] Referring now to FIG. 12A, there is shown an example 3D specimen scanning system 1200 for generating a 3D representation of a system according to an illustrative embodiment. In various examples, system 1200 may be configured to implement or carry out method 1100 as illustrated in FIG. 11 (and also FIGS. 14A and 14B referred to below).

[0184] In this example, 3D specimen scanning system 1200 comprises a multi-specimen loader 1210 that is configured to load a plurality of mounted specimens for scanning, wherein each mounted specimen is loaded into a corresponding specimen receiving location of the multi -specimen loader 1210. Scanning system 1200 further comprises a transport mechanism 1230 for transporting a selected mounted specimen from the multi-specimen loader 1210 to a scanning location and back to the multi-specimen loader 1210 following any 3D scanning and a 3D scanner 1240 for 3D scanning the selected mounted specimen and generating a 3D representation of the selected specimen.

[0185] Referring now to FIG. 12B, there is shown a system overview diagram of a 3D scanner which in one example may comprise 3D scanner 1240 illustrated in FIG. 12A. In this example, 3D scanner 1240 comprises a camera 1241 for capturing respective 2D camera images of the selected mounted specimen in a plurality of different specimen orientations with respect to the camera and an orientation mechanism 1242 for orienting the mounted specimen to the plurality of different specimen orientations. 3D scanner 1240 further comprises a 3D representation processor 1243 comprising one or more data processors for generating the 3D representation of the mounted specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations. In various examples, 3D representation processor 1243 may be a computing device or system such as image processor 220 shown in FIG. 2A and / or 3D representation processor 243 shown in FIG. 2B.

[0186] In one example, transporting the selected mounted specimen to and from the multispecimen loader (ie, steps 1120 and 1140 of FIG. 11) comprises transferring the selected mounted specimen from a fixed specimen transport location corresponding to a specimen receiving location of themulti-specimen loader. In one example, following 3D scanning of the selected mounted specimen a new mounted specimen is advanced to the fixed specimen transport location for subsequent 3D scanning. In various examples, the multi-specimen loader is configured to arrange the corresponding specimen receiving locations in a queue and advancing a next mounted specimen to the fixed specimen transport location for 3D scanning comprises moving the multi-specimen loader to move or advance the next mounted specimen in the queue to the specimen transport location for scanning.

[0187] Referring now to FIGS. 13A and 13B, there are shown figurative diagrams of example multi-specimen loaders 1300 and 1350 which in one example may correspond to the multi -specimen loader 1210 shown in FIG. 12. Consider multi-specimen loader 1300 shown in FIG. 13A comprising an elongate movable platform 1320 where the specimen receiving locations (ie, specimen receiving locations 1-7) are arranged in a queue in the form of a row of specimen receiving locations extending along elongate platform 1220. In this example, a linear queue or row of seven specimen receiving locations is shown and the specimen transport location 1310 is indicated as the shaded location and corresponds to receiving location 4 (left hand side) prior to the elongate platform 1320 of multi -specimen loader 1300 moving linearly as indicated. Following movement of platform, the specimen transport location 1310 now corresponds to receiving location 5 of the multi-specimen loader 1300 (right hand side) and the next mounted specimen in the queue is now ready for scanning. In this example, a linear or elongate multispecimen loader may have its specimen receiving locations loaded and the 3D specimen scanning system then operated to 3D scan all the mounted specimens in the linear multi-specimen loader by advancing the elongate platform 1320 without operator intervention.

[0188] By contrast, multi-specimen loader 1350 shown in FIG. 13B comprises a continuous queue of specimen receiving locations (ie, specimen receiving locations 1-8) where the specimen receiving locations are arranged around the periphery of circular platform 1370. In this example, the specimen transport location 1360 is indicated as the shaded location and corresponds to receiving location 4 (left hand side) prior to the rotation of circular platform 1370 of multi-specimen loader 1350 where the specimen transport location 1360 now corresponds to receiving location 5 of multi-specimen loader 1350 (right hand side) and the next mounted specimen in the queue is now ready for scanning.

[0189] As would be appreciated, a multi-specimen loader comprising a platform 1320 having a circular configuration such as a carousel or rotatable platform may be operated in a continuous operating mode where an operator unloads a scanned specimen when the multi -specimen loader is stationary (eg, during scanning of another mounted specimen) and reloads the now vacant specimen receiving location with a new specimen which will eventually be scanned following rotation of the carousel as the remaining mounted specimens are 3D scanned.

[0190] As can be seen, in various embodiments the multi-specimen loader is configured to queue the plurality of mounted specimens for scanning and may comprise a movable platform that moves to sequentially present mounted specimens at the specimen transport location fortransporting and scanning.

[0191] In one example, each mounted specimen further comprises a transport mount to which the specimen mount is removably attached and the specimen receiving locations of the multi -specimen loader are configured to receive the transport mounts of the plurality of mounted specimens. As would be appreciated, specimen collections often comprise specimens that are already mounted (ie, include a specimen mount). In this manner, the specimen mount may be attached to the transport mount prior to loading of the multi-specimen loader and manipulation and transport of the mounted specimen occurs by manipulating and transporting the transport mount which reduces any potential impact to the specimen from the scanning process.

[0192] Consider the non-limiting example of insect collections as referred to above where the specimen mount comprises a pin that extends through the specimen so that insect specimen is located approximately mid-way down the pin. As noted above, this type of mounting arrangement is common for mounting delicate and fragile biological specimens and removing the pin from the insect is likely to cause damage to the specimen. In one example, the transport mount would comprise (as an example) a base member and top mounting portion to which the end of the pin could be removably mounted to and where the specimen receiving locations of the multi-specimen loader would be configured to receive the base members of the transport mounts. In this example, the insect specimen may be transported and scanned (which may include reorientation of the insect specimen) through manipulation of the transport mount as a result minimising the risk of damaging the specimen.

[0193] In one example, the transport mount comprises a magnetic material (ie, either a magnet or material which is attracted to a magnet) that is magnetically attracted to a complementary magnetic material located at the specimen receiving location and / or scanning location to assist in positively positioning (eg, locating and / or orienting) the mounted specimen at the specimen receiving location and / or scanning location. In one example, the magnetic material located at the specimen receiving location and / or scanning location may be a switchable electromagnet which may be turned on or off when a mounted specimen is required to be transported between the different locations. In another example, the magnetic material is located in a base region of the transport mount and the complementary magnetic material is located in a floor or support region at the specimen receiving location and / or scanning location.

[0194] Referring back to FIG. 11, at steps 1120 and 1140 the specimen is transported from the multi-specimen loader to the scanning location for seaming and then back to the multi-specimen loader following 3D scanning. Referring now to FIG. 14A, there is shown a flow diagram of an example method1400 for transporting a mounted specimen to and from a multi -specimen loader in accordance with some embodiments. At block 1410, a transport mechanism is releasably attached to the transport mount of the mounted specimen that is to be 3D scanned. At block 1420, the mounted specimen is transported to the scanning location for 3D scanning and then at block 1430 the mounted specimen is then transported back from the scanning location to the multi -specimen loader following 3D scanning. Finally, at block 1440 the transport mechanism is detached from the transport mount of the mounted specimen. In one example, the next mounted specimen in the queue is then advanced for 3D scanning.

[0195] As would be appreciated, transporting the selected mounted specimen loader to the scanner loader may be achieved by any suitable electromechanical transport mechanism that functions to transport, transfer or move the mounted specimen form the multi-specimen loader to the scanning location for 3D scanning. In one example, where there is a defined specimen transport location on the multi-specimen loader where the specimen is transported to and from, the transport mechanism may comprise a relatively simple mechanism that functions to removably attach to the mounted specimen, and in one example to a transport mount of the mounted specimen, transport the mounted specimen to the scanning location where the mounted specimen is 3D scanned, and then return the mounted specimen to the specimen transport location on the multi-specimen loader. The multi-specimen loader is then allowed to advance so that the next mounted specimen may be transported in the same manner. Depending on the implementation, the transport mechanism may detach from the mounted specimen once the mounted specimen is at the scanning location.

[0196] Some examples of transport mechanisms that may be employed in accordance with various embodiments include a robotic arm having an operative end that is configured to removably attach to the mounted specimen. As discussed above, in various examples the mounted specimen may be in the form of a specimen mount to which the specimen is permanently attached to and a second transport mount to which the specimen mount is removably attached. In these examples, the transport mechanism may be configured to removably attach to the transport mount and transport or move the combined specimen mount and transport mount arrangement.

[0197] In one example, the transport mechanism may comprise a connecting portion that removably attaches to a complementary connecting portion of the mounted specimen. In one example, the connecting portion may comprise a gripper that is operable to grip the mounted specimen (eg, pinned specimen). In examples, where the mounted specimen comprises a transport mount, the complementary connecting portion is located on the transport mount and the connecting portion of the transport mechanism removably attaches to the transport mount. In one example, the connecting portion of the transport mechanism attaches to the mounted specimen when moving in an attaching direction with respect to the mounted specimen and detaches or releases from the mounted specimen when moving in a detaching direction with respect to the mounted specimen.

[0198] In one example, the transport mechanism comprises a loading arm configured to releasably attach to the transport mount when moving in a first loading direction with respect to the transport mount and to detach from the transport mount when moving in an unloading direction with respect to the transport mount.

[0199] In one example, the loading arm is rotatable to releasably attach to the transport mount when rotating in the loading direction and to detach when rotating in the unloading direction. In one example, the loading arm rotates in a horizontal plane where the loading arm comprises a connecting portion that removably connects to a complementary connecting portion of the mounted specimen. In this example transport mechanism, the loading arm rotates about one end in a loading movement and loads the mounted specimen from specimen transport location, with the loading arm further rotating to transport the mounted specimen to the scanning location where the mounted specimen is then placed for 3D scanning. In one example, where the mounted specimen remains stationary while the camera moves with respect to the mounted specimen to capture respective 2D images at different orientations, the loading arm may remain attached or connected to the mounted specimen and then continue to rotate in the same direction to transport the mounted specimen back to the specimen transport location on the multi-specimen loader following scanning.

[0200] Referring back to FIG. 11, at block 1130 the selected mounted specimen is 3D scanned at the scanning location to generate a 3D representation of the selected specimen. Referring now to FIG. 14B, there is shown is a flow diagram of an example method 1450 of 3D scanning a selected mounted specimen in accordance with some embodiments. In one example, method 1450 may be implemented by 3D specimen scanning system 1200 illustrated in FIG. 12A.

[0201] At block 1460, method 1450 comprises orienting the mounted specimen to a plurality of different specimen orientations with respect to a camera configured to generate 2D images. As would be appreciated, 3D cameras may be generally configurable to generate 2D images.

[0202] In one example, the specimen is oriented (eg, by rotation) to adopt the two or more different specimen orientations with respect to the scanning camera while the location and orientation of the camera is fixed. In another example, the mounted specimen remains stationary, and the camera is reoriented with respect to the stationary specimen to generate the two or more different orientations with respect to the camera. In yet another example, both the orientation of the specimen and the camera may be changed relative to each other in order to achieve the different specimen orientations with respect to the camera.

[0203] In one example, an orienting mechanism for orienting the selected mounted specimen with respect to the camera may comprise a specimen orientation arrangement for changing or modifyingthe specimen orientation and / or a camera orienting arrangement for changing or modifying the camera orientation. In one example, the specimen orientation arrangement is configured to rotate the mounted specimen about a vertical axis and / or change the tilt or elevation angle of the mounted specimen. In other examples, the vertical axis is substantially through the centre of the specimen in which case rotation of the specimen corresponds to a variation in azimuthal angle for the specimen. In one example, the selected mounted specimen is rotated 360 degrees in fixed increments (eg, 5 degree) for a given camera position, and the camera is then rotated about the specimen and specimen again rotated through 360 degrees. In another example, the rotation angles (in any axis) are selected to maintain a uniform pose density with respect to the specimen. In one example, the rotation angles are selected to maintain the geodesic distance between the camera and the specimen.

[0204] In another example, the different orientations may be selected based on the geometry or configuration of the selected mounted specimen to be scanned. As an example, the different orientations adopted for the situation where the camera is viewing a region of the specimen having more variation in the surface geometry may be selected to be more closely spaced or denser in contrast to where the camera is viewing a region of the specimen where the surface geometry does not vary considerably. In one example, the different orientations may be selected to follow the surface profile so as to maintain a substantially constant distance between the camera and the region of the specimen being viewed.

[0205] In one example, camera orientation arrangement is configured to rotate or more generally move the camera about the mounted specimen (eg, by the use of a robotic arm configured to point the camera at the mounted specimen and move the camera to different locations at a constant distance from the specimen).

[0206] In one example, camera orientation arrangement is configured to rotate the camera about an axis extending through the specimen such as by attaching the camera to a rotating member centred at the specimen. In one example, where the axis of rotation is horizontal and extends substantially through the centre of the specimen then rotation of the camera will correspond to a variation in elevation angle. In other examples, the camera may be attached at various radial locations on the rotating member to change the distance between the camera and the mounted specimen.

[0207] As would be appreciated, achieving reorientation of the mounted specimen with respect to the camera by modifying the orientation of the camera where possible assists in reducing the amount of unwanted movement of the specimen which may be a concern where the specimen is delicate or fragile.

[0208] At block 1470, respective 2D camera images of the mounted specimen are captured corresponding to the plurality of different specimen orientations.

[0209] At block 1480, 3D representation of the specimen is generated based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

[0210] In one example, a multi -view stereo method is used to generate the 3D representation from the multiple 2D images and associated pose information (eg, multi-view stereo pipeline of COLMAP). In another example, a neural radiance field (NeRF) based method is adopted to generate the 3D representation from the multiple 2D images and associated pose information (eg, nerfstudio).

[0211] Other example methods for generating a 3D representation that may be used in accordance with the present disclosure include, but are not limited to, NeRF, Mip-NERF, Neus, Nerfacto, Nerfacc, gaussian splatting or Instant-NG.

[0212] In one example, the pose information is determined from the known relative positioning and orientation of the camera with respect to the specimen. In another example, the pose information is determined from the camera image. In one example, the pose information is determined from the camera image using a structure-from-motion method (eg, structure -from-motion pipeline from COLMAP or Alicevision Meshroom). In another example, the pose information is determined from both the relative positioning and orientation of the camera and any estimate determined from the camera image. In one example, the camera image is pre-processed to remove any background artefacts. In one example, background subtraction was carried out employing OpenCV’s morphologyEx and the Rembg Python module using a clean background without a specimen.

[0213] In one example, the 3D representation is in the form of a 3D mesh. In another example, the 3D mesh is generated by Poisson surface reconstruction method that functions to convert a point cloud adopting a smoothness prior. In another example, a Neural Kernel Field approach may be adopted that uses a data driven prior along with the smoothness prior.

[0214] In another example, 3D scanning the selected specimen comprises orienting the mounted specimen to a plurality of different specimen orientations with respect to a camera or sensor configured to generate 3D data images. In one example, the camera may be an RGBD camera that also generates 3D depth data. In another example, the camera may comprise a laser scanner (eg, LiDAR and / or line profile scanner) that generates a 3D data image in the form of a point cloud. Respective 3D data images of the mounted specimen corresponding to the plurality of different specimen orientations are then captured and a 3D representation of the specimen may be generated based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

[0215] In one example, where the camera comprises a RGBD camera, a visual odometry based method or approach is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, KinectFusion). In another example, a simultaneous localization and mapping (SLAM) based method is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, ORB-SLAM2, BAD-SLAM, or Gradient-SDF).

[0216] In another example, where the camera comprises a laser scanner, a point cloud registration from motion control approach (eg, see Wang., 2021, “3D Reconstruction Using a Linear Laser Scanner and a Camera”, in 2021 2nd International Conference on Artificial Intelligence and Computer Engineering (ICAICE), pp. 666-673, IEEE) is used to generate the 3D representation from the multiple 3D data images and associated pose information. In another example, a SLAM based method is used to generate the 3D representation from the multiple 3D data images and associated pose information (eg, Visual SLAM or WildCAT).

[0217] In various examples, the 3D representation generated from the multiple 3D data images may be in the form of including, but not limited to, a 3D point cloud, a 3D mesh, a Signed Distance Field (SDF) that may in turn be interrogated to extract a 3D mesh, or a deep network encoding the 3D representation.

[0218] Referring now to FIGS. 15A and 15B there are shown front and rear perspective views of a 3D specimen scanning system 500 in accordance with some embodiments. In this example, 3D specimen scanning system 1500 comprises a multi-station loader 1510 for loading a plurality of mounted specimens 1515 that are loaded into a corresponding specimen receiving location 1516. 3D specimen scanning system 500 further comprises a transport mechanism 1520 for transporting a selected mounted specimen 1518 from multi-station loader 1510 to a scanning location 1525 and back to loader 1510 following any 3D scanning and a 3D scanner 1530 for 3D scanning the selected mounted specimen and generating a 3D representation.

[0219] FIG. 16 is an end exploded view (laterally) showing the different system components of scanning system 1500.

[0220] Referring now to FIG. 17, there is shown a perspective view of a mounted specimen 1700 comprising a mounting arrangement configured to allow both transport and 3D scanning of the specimen 1710. In various examples, mounted specimen 1700 may correspond to the mounted specimens shown in FIGS 15A, 15B and 16 (eg, 1515, 1518)

[0221] In this example, the mounted specimen comprises two mounting components including a specimen mount 730 to which the specimen 1710 is mounted and then a transport mount 1750 to whichthe specimen mount 1730 is then removably mounted or attached to. In this example, specimen mount 1730 comprises a pin 1735 that extends through specimen 1710 so that insect specimen is located approximately mid-way down the pin 1735. As has been observed previously, this type of specimen mount is common for mounting delicate and fragile biological specimens and removing the pin 1735 from the insect is likely to cause damage to the specimen.

[0222] Transport mount 1750 in this example comprises a cylindrical shaped base 1751 and an upwardly extending tapered conical portion 1755 that includes a top aperture 1756 to removably receive (in this example) the pin 1735 of specimen mount 1730. Located between base 1751 and tapered conical portion 1755 is a necked engagement region 1753 for releasable attachment with the transport mechanism to allow transport of the mounted specimen to and from the scanning location as will be discussed below. In this example, cylindrical base 1751 includes a magnetised material 1760 in the form of a permanent magnet located on its underside which will removably magnetically attach to a corresponding magnetic material in the form of a permanent magnet formed in the base of specimen receiving location of the multi-station loader 1510 and at the support location. This functions to positively position the transport mount (ie, locate and orient) at these various locations.

[0223] Referring now to FIGS 18A and 18B there are shown perspective and top views of a multi-specimen loader 1800 which in one example corresponds to multi -station loader 1510 shown in FIGS 15A, 15B and 16.

[0224] In this example, multi-station loader 1800 comprises a circular platform or carousel 1810 (see also FIG 13B) having a six specimen receiving locations 1815A-F spaced around the periphery of carousel 1810 (as best seen in FIG. 18A) and a centrally disposed axial motor 1830 configured to rotate multi-specimen loader 1800 about a central vertically extending rotation axis. In this example, multispecimen loader 1800 further comprises a stand or support to raise the carousel 1810 to an elevated position with respect to “ground” level.

[0225] In this example, the specimen transport location 1817 (corresponding to specimen receiving location 1815F) where the mounted specimen is transported to and from is shown as vacant but other specimen receiving locations 1815A-E each have a loaded mounted specimen 1700. Each specimen receiving location comprises an arcuate guide channel 1820 (as can be seen at specimen transport location 1817) having a width corresponding the diameter of cylindrical base 1751 of mounted specimen 1700. Guide channel 1820 guides the movement of mounted specimen 1700 during transport to and from the scanning location. Furthermore, each specimen receiving location includes a magnetised region or magnetic material 1860 that is magnetically attracted to the permanent magnet 1760 in the base 1751 mounted specimen 1700.

[0226] Referring now to FIGS. 19A and 19B, there are shown perspective and top views of a transport mechanism 1900 which in one example corresponds to transport mechanism transport mechanism 1520 shown in FIGS 15A, 15B and 16. Transport mechanism 1900 in this example comprises a loading arm 1910 comprising a connecting portion 920 that removably attaches or engages with a complementary attachment portion of the mounted specimen 1700. In this example, attachment portion 920 comprises a slotted region 921 which removably engages with the necked attachment portion 1753 of the transport mount 1750 of mounted specimen 1700. Transport mechanism 1900 further comprises a motor 1930 configured to rotate loading arm 1910 about a vertically extending axis at one end and a stand or support 1950 to raise the loading arm 1910 to an elevated position with respect to “ground” level and at a commensurate level to that of the specimen transport location of an associated multi-station loader 1810 (eg, see FIG. 16).

[0227] FIG. 19B shows the stages for transporting mounted specimen 1700 from the multispecimen loader to a scanning location and back to the multi-specimen loader following any 3D scanning. Mounted specimen 1700 is shown on the left hand side as being collected from specimen transport location through clockwise movement of loading arm 1910 in a loading direction where the slotted region 1921 of connecting portion 920 will rotate and engage with the necked connecting portion 1753 of mounted specimen 1700 and transport the mounted specimen 1700 in a clockwise direction until it reaches the scanning location as shown in the right hand side view. At this stage, loading arm 1910 can then retract in an anti -clockwise direction in an unloading direction leaving mounted specimen 1700 at the scanning location. Once 3D scanning has been completed, loading arm 1910 will once again move in a clockwise direction and collect mounted specimen and continue to rotate until the mounted specimen 1700 is aligned with specimen transport location and the loading arm 1910 will then retract in an anticlockwise direction to await a new mounted specimen to be positioned at the specimen transport location following movement of the multi-specimen loader.

[0228] In this transport mechanism, the loading arm in a loading movement loads the mounted specimen from specimen transport location, with the arm then moving to the scanning location where the mounted specimen is then placed for 3D scanning. In one example, where the mounted specimen remains stationary while the scanning camera moves with respect to the mounted specimen to capture respective 2D images at different orientations, the loading arm may remain attached or connected to the mounted specimen and then swing back to transport the mounted specimen back to the specimen transport location on the multi-station loader following scanning.

[0229] In one example, the base of the transport mount comprises a magnetic material which functions to assist in locating the mounted specimen at the specimen receiving location and the scanning location which themselves include a support portion formed of a magnetic material to which the base will attach to. This not only assist in positively locating the mounted specimen in a selected location but alsoin preserving the orientation of mounted specimen (eg, in an upright orientation) which assists in both the transporting and scanning process.

[0230] Referring now to FIG. 20, there is shown an exploded perspective view of a 3D scanner 2000 which in one example corresponds to 3D scanner 1530 shown in FIGS 15A, 15B and 16. In this example, 3D scanner 2000 comprises a scanning camera 2010 for capturing 2D images and an orientation mechanism 2030 for orienting the specimen to a plurality of different specimen orientations with respect to the scanning camera 2010 and a 3D representation processor 2060 or computing device for generating the 3D representation of the specimen based on the captured 2D images and associated pose information.

[0231] In this example, orientation mechanism 2030 comprises a specimen orientation arrangement 2040 for changing or modifying a specimen orientation and a camera orientation arrangement 2050 for changing or modifying the camera orientation. In this example, specimen orientation arrangement 2050 is configured to rotate the mounted specimen 1700 about a vertical axis at the scanning location as indicated (ie, different azimuth angles). In this example, specimen orientation arrangement 2040 comprises a rotatable base 2041 corresponding to the scanning location and a motor 2042 controllable to rotate base 2041 and mounted specimen 1700 at the scanning location. In various examples, rotatable base 2041 may include a magnetic material region which is magnetically attracted to the magnetic material 1760 in the base 1751 of mounted specimen 1700. Specimen orientation arrangement 2040 further comprises a stand or support 2043 to raise the scanning location 1910 to an elevated position with respect to “ground” level and at a commensurate level to that of the specimen transport location of an associated multi-station loader 1810 (eg, see FIG. 16).

[0232] Camera orientation arrangement 2050 is configured to rotate camera 2010 about a horizontal axis extending substantially through the mounted specimen as indicated (ie, different elevation angles) and in this example comprises an upright rotating circularly shaped mount or wheel 2055 to which the camera 2010 is attached to and oriented towards the centre of rotation of wheel 2055 and a motor (not shown) controllable to rotate wheel 2055. In one example, a counterweight may be attached to wheel 2055 to ensure smooth movement and reduce motor torque required to cause movement. As would be appreciated, the elevation angle could be varied by changing the tilt angle of rotatable base 2041 (as an example), however, for delicate or fragile specimens this may potentially damage the specimen. A camera orientation arrangement in accordance with FIG. 20 will limit any unwanted movement of the specimen that may result from the effects of gravity or vibration which may affect the generation of the 3D representation which generally assumes that appearance of the specimen will be invariant to image perspective.

[0233] In this example, camera 2010 may be positioned at different radial distances from the centre of rotation to vary the distance between the camera and the mounted specimen along an axisdefined by the viewing direction of the camera. In various examples, cameras having different focal lengths may be adopted and the ability to change the distance between the mounted specimen and the camera may be advantageous. In one example, camera 2010 may have a focal length of 35 mm. In another example, camera 2010 may have a focal length of 16 mm. In various examples, camera 2010 may have a 12MP resolution. Note that FIG. 20 is an exploded view, the mounted specimen 1700 would be placed closer to wheel 2055 (as indicated by arrow) to be viewed by camera 2010 (eg, see FIGS 15A and 15B).

[0234] In operation, the specimen orientation arrangement 2040 and / or camera orientation arrangement 2050 are commanded to adopt a selected azimuth and elevation angle respectively with respect to the mounted specimen. In one example, each of the azimuth angle and elevation angles are increased or decreased by a constant rotation step (eg, 10 degrees). Referring now to FIG. 21, there is shown a plot 2100 of estimated camera poses captured by 2D camera (35 mm focal length) where the elevation angle has been varied from approximately -40 degrees to +40 degrees (defined with respect to a horizontal plane) in 10 degree steps and the azimuth angle has been varied between -180 degrees to +180 degrees (ie, full 360 degree rotation) similarly in 10 degree steps. In this example, this would correspond to 324 2D images being captured for a given 3D specimen scanning where the specimen is located at the centre of rotation (in both axes) to avoid translation relative to the image plane so that the entire specimen is visible and the camera pose is accurate for all viewing angles.

[0235] 3D representation processor 2060 then functions to generate a 3D representation of the selected specimen based on the captured 2D images and the associated pose information corresponding to the different specimen orientations as has been previously described. As would be appreciated, while the pose for a given 2D image and the pose information for multiple 2D images may be determined by pose estimation (eg, COLMAP), in other examples the camera pose with respect to the mounted specimen may be determined in accordance with the position and orientation as determined by the specimen and camera orientation arrangements. In one example, the motor controller values used to control the specimen and camera orientation arrangements to their desired orientations may be employed to derive the camera pose with respect to the mounted specimen. In other examples, the orientations of the specimen and camera orientation arrangements may be directly measured, such as by the use of angular encoders, and these measured orientations may be employed to derive the camera pose with respect to the mounted specimen. In yet other examples, the pose may be estimated and then checked against an expected pose based on the orientation expected from the specimen and camera orientation arrangements.

[0236] In another example, scanning camera 310 comprises a camera or sensor configured to generate 3D data images such as an RGBD camera that also generates 3D depth data or a laser scanner (eg, LiDAR and / or line profile scanner) that generates a 3D data image in the form of a point cloud. Inthis example, 3D representation processor 2060 then functions to generate a 3D representation of the selected specimen based on the captured 3D data images.

[0237] Referring now to FIG. 22, there is shown the images of five specimens (ROW A) and the associated 3D representations based on:• ROW B - 3D point cloud reconstructions from a stereo-view model (COLMAP),• ROW C- 3D mesh reconstructions from a stereo-view model (COLMAP), and• ROW D - 3D mesh reconstructions from a neural radiance field model (NeRFacto); all generated in accordance with embodiments of the present disclosure.

[0238] As can be seen from inspection, the generated 3D representations comprise a high level of detail and include minute details such as insect antennae. The missing representation for the fifth specimen at ROW D is due to a failure of convergence of the NeRF model. In these examples, for the COLMAP based 3D representation generating process the 2D images were downsized to a resolution of 2028 x 1520 pixels while for the NeRFacto based 3D representation generating process the 2D images were downsized to a resolution of 1014 x 760 pixels to reduce processing time.

[0239] As can be appreciated, a multi-specimen loader based 3D specimen scanning method and system in accordance with the disclosure may allow for continuous loading and unloading insect specimens providing a high throughput specimen scanning capability. Additionally, images of the specimen may be captured over a wide range of angles. In accordance with various embodiments, fragile or delicate mounted specimens such as insect specimens will be kept upright during scanning to avoid the risk of insect moving on the pin.

[0240] As can be seen from above, the methods and systems described in accordance with the present disclosure function to automate both the specimen handling and 3D scanning processes as a result reducing the requirement for trained human operators to carry out the scanning task.

[0241] Individual aspects of the present disclosure may be described as a process or method which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process is terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function.

[0242] The various illustrative logical blocks, modules, circuits, and algorithm steps described in various aspects of the present disclosure may be implemented as electronic hardware, computer software, firmware, or combinations thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0243] Devices implementing processes and methods according to these disclosures can include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and can take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks (e.g., a computer-program product) may be stored in a computer-readable or machine -readable medium. A processor(s) may perform the necessary tasks. Typical examples of form factors include laptops, smart phones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rackmount devices, standalone devices, and so on. Functionality described herein also can be embodied in peripherals or add-in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example.

[0244] The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that such prior art forms part of the common general knowledge.

[0245] It will be understood that the terms “comprise” and “include” and any of their derivatives (eg, comprises, comprising, includes, including) as used in this specification, and the claims that follow, is to be taken to be inclusive of features to which the term refers, and is not meant to exclude the presence of any additional features unless otherwise stated or implied.

[0246] In some cases, a single embodiment may, for succinctness and / or to assist in understanding the scope of the disclosure, combine multiple features. It is to be understood that in such a case, these multiple features may be provided separately (in separate embodiments), or in any other suitable combination. Alternatively, where separate features are described in separate embodiments, these separate features may be combined into a single embodiment unless otherwise stated or implied. This also applies to the claims which can be recombined in any combination. That is a claim may be amended to include a feature defined in any other claim. Further a phrase referring to “at least one of’ a list of itemsrefers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c.

[0247] It will be appreciated by those skilled in the art that the disclosure is not restricted in its use to the particular application or applications described. Neither is the present disclosure restricted in its preferred embodiment with regard to the particular elements and / or features described or depicted herein. It will be appreciated that the disclosure is not limited to the embodiment or embodiments disclosed, but is capable of numerous rearrangements, modifications and substitutions without departing from the scope as set forth and defined by the following claims.

[0248] Please note that the following claims are provisional claims only, and are provided as examples of possible claims and are not intended to limit the scope of what may be claimed in any future patent applications based on the present application. Integers may be added to or omitted from the example claims at a later date so as to further define or re-define the scope.

Claims

CLAIMS1. A 3D specimen scanning system comprising: a multi-specimen loader for loading a plurality of mounted specimens for scanning, wherein the plurality of mounted specimens are loaded into corresponding specimen receiving locations of the multispecimen loader, wherein each mounted specimen comprises a specimen and a specimen mount; a transport mechanism for transporting a selected mounted specimen from the multi-specimen loader to a scanning location and back to the multi-specimen loader following any 3D scanning; and a 3D scanner for 3D scanning the selected mounted specimen and generating a 3D representation of the selected specimen.

2. The 3D specimen scanning system of claim 1, wherein the transport mechanism is configured to transport the selected mounted specimen from a fixed specimen transport location corresponding to a specimen receiving location of the multi-specimen loader.

3. The 3D specimen scanning system of claim 2, wherein the multi-specimen loader is configured to advance a next mounted specimen to the fixed specimen transport location for 3D scanning.

4. The 3D specimen scanning system of claim 3, wherein the corresponding specimen receiving locations are arranged in a queue and the multi -specimen loader moves to advance the next mounted specimen in the queue to the specimen transport location for scanning.

5. The 3D specimen scanning system of claim 4, wherein the multi-specimen loader comprises a platform having a circular configuration and wherein the queue of corresponding receiving locations is arranged around a periphery of the multi-specimen loader and wherein moving the multi -specimen loader comprises rotating the multi-specimen loader.

6. The 3D specimen scanning system of any preceding claim, wherein each mounted specimen further comprises a transport mount to which the specimen mount is removably attached to and wherein the specimen receiving locations are configured to receive the transport mounts of the plurality of mounted specimens.

7. The 3D specimen scanning system of claim 6, wherein the transport mount comprises a magnetic material that is magnetically attracted to a complementary magnetic material located at the specimen receiving location and / or scanning location to positively position the mounted specimen at the specimen receiving location and / or scanning location.

8. The 3D specimen scanning system of any one of claims 6 or 7, wherein the transport mechanism is configured to: releasably attach to the transport mount of the selected mounted specimen; transport the selected mounted specimen to the scanning location for 3D scanning; transport the selected mounted specimen back from the scanning location to the multi-specimen loader following 3D scanning; and detach from the transport mount of the selected mounted specimen.

9. The 3D specimen scanning system of claim 8, wherein the transport mechanism comprises a loading arm configured to releasably attach to the transport mount when moving in a first loading direction with respect to the transport mount and to detach from the transport mount when moving in an unloading direction with respect to the transport mount.

10. The 3D specimen scanning system of claim 9, wherein the loading arm is rotatable to releasably attach to the transport mount when rotating in the loading direction and to detach when rotating in the unloading direction.

11. The 3D specimen scanning system of any preceding claim, wherein the 3D scanner comprises: a camera for capturing respective 2D camera images of the selected mounted specimen in a plurality of different specimen orientations with respect to the camera; an orientation mechanism for orienting the selected mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the selected specimen based on the respective 2D camera images and associated pose information corresponding to the plurality of different specimen orientations.

12. The 3D specimen scanning system of claim 11, wherein the orientation mechanism comprises: a specimen orientation arrangement for changing a specimen orientation; and / or a camera orientation arrangement for changing a camera orientation.

13. The 3D specimen scanning system of claim 12, wherein the specimen orientation arrangement is configured to rotate the selected mounted specimen about a vertical axis substantially extending through the mounted specimen.

14. The 3D specimen scanning system of claim 12 or 13, wherein the camera orientation arrangement is configured to rotating the selected mounted specimen about a horizontal axis substantially extending through the mounted specimen.

15. The 3D specimen scanning system of any one of claims 11 to 14, wherein the 3D representation processor is configured for generating a 3D representation of the specimen based on the respective camera images and associated pose information by a multi -view stereo method.

16. The 3D specimen scanning system of any one of claims 11 to 14, wherein the 3D representation processor is configured for generating a 3D representation of the specimen based on the respective camera images and associated pose information by a neural radiance field based method.

17. The 3D specimen scanning system of any one of claims 11 to 16, wherein the associated pose information is determined by the 3D representation processor from a structure from motion analysis of the respective 2D camera images.

18. The 3D specimen scanning system of any one of claims 11 to 16, wherein the associated pose information is determined by the 3D representation processor from the relative positioning and orientation of the camera with respect to the specimen.

19. The 3D specimen scanning system of any preceding claim, wherein the 3D scanner comprises: a camera for capturing respective 3D data images of the selected mounted specimen in a plurality of different specimen orientations with respect to the camera; an orientation mechanism for orienting the selected mounted specimen to the plurality of different specimen orientations; a 3D representation processor comprising one or more data processors for generating the 3D representation of the selected specimen based on the respective 3D data images and associated pose information corresponding to the plurality of different specimen orientations.

20. A method comprising: loading a plurality of mounted specimens into corresponding specimen receiving locations of a multi-specimen loader, wherein each mounted specimen comprises a specimen and a specimen mount; transporting a selected mounted specimen from the multi-specimen loader to a scanning location; 3D scanning the selected mounted specimen at the scanning location to generate a 3D representation of the selected specimen; and transporting the selected mounted specimen from the scanning location to the multi -specimen loader following 3D scanning.

Images