Device and method for processing and cataloguing samples

Abstract

An apparatus for processing and cataloging a sample in a container having a label with initial identification data, the apparatus having a frame; a plurality of cameras mounted to the frame; and a controller system coupled to the cameras, the controller system comprising a controller and a non-transitory memory storing instructions that, when executed by the controller, cause computer processing to be performed to produce a cataloging record output characterizing the sample, the record also including the identification data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 694,093, filed on July 5, 2018, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to apparatus and methods for the processing and cataloguing of samples, and more particularly to samples of tissue specimens (typically in research and clinical laboratories) and items obtained from crime scenes for evidentiary or investigative purposes. Background Art

[0004] Laboratory work involves the modification and analysis of tissues and other biological specimens through a variety of medical and research processes, in many cases in large volumes. These processes often evaluate specimens of varying size and detail, and require accurate and consistent tracking of specimen details and recording of the necessary work performed on said specimens. In many cases, the specimens under investigation are unique and cannot be replaced or duplicated if lost, mislabeled, or mishandled. Given the nature of the research and / or clinical work being conducted, detailed and highly accurate records and specimen measurements and tracking are required; this work often requires a high degree of precision and is repetitive in nature. As such, this work may strain the human technician's ability to be accurate and precise. Summary of the invention

[0005] According to one embodiment of the present invention, there is provided an apparatus for processing and cataloguing samples in containers having labels with initial identification data. In this embodiment, the apparatus comprises: a frame defining a working area, wherein the working area comprises locations for receiving the containers. The apparatus further comprises a plurality of cameras mounted to the frame. The apparatus further comprises a controller system coupled to the cameras, the controller system comprising a controller and a non-transitory memory storing instructions, the instructions, when executed by the controller, causing a computer process to be performed. In this embodiment, the computer process comprises:

[0006] causing a first camera among the cameras to capture an image of a sample in a working area;

[0007] causing a second camera in the cameras to capture an image of the identification data;

[0008] processing the image of the sample into a data output representing a property of the sample;

[0009] integrating the data output into a catalogued record output representing the sample, the record also including identification data; and

[0010] The output of catalogued records is transferred over a network to a system of record for storage.

[0011] Optionally, processing the image of the sample comprises forming a composite image comprising an image of the identification data and an image of the sample. Optionally, causing the first of the cameras to capture and causing the second of the cameras to capture occurs in a manner wherein the capturing by each camera occurs substantially simultaneously. Alternatively or additionally, the cataloguing record output comprises an image of the sample, an image of the identification data, and a timestamp indicating the time at which the image of the identification data and the image of the sample were captured.

[0012] Optionally, the sample is to be transferred to a processing cartridge having second identification data thereon, and the working area has a position for accommodating the cartridge while also accommodating the container in the working area, and the computer processing performed by the controller further comprises:

[0013] causing one of the cameras to capture an image of the second identification data;

[0014] converting the initial identification data and the second identification data into a first digitally encoded text and a second digitally encoded text, respectively; and

[0015] The first digitally encoded text and the second digitally encoded text are compared for consistency.

[0016] Alternatively or additionally, the computer processing also includes causing a selected one of the cameras to capture an image set, the image set recording the transfer of the sample from the container to the processing box. Alternatively or additionally, the captured image set is a video record. Alternatively or additionally, the computer processing also includes analyzing the captured image set, the image set recording the transfer, to identify the obvious deviation in the activity compared to the activity associated with the normal transfer. Alternatively or additionally, the deviation is a deviation that a part of the sample to be transferred looks like it has been modified or lost. Alternatively or additionally, the deviation is a deviation that a part of the sample to be transferred has been retained in the container. Alternatively or additionally, the deviation is based on the second digital code text and the first digital code text matching failure, the sample looks like it has been transferred to the incorrect processing box. Alternatively or additionally, the computer processing also includes: evaluating the data output of the attribute characterizing the sample to meet the minimum standard for downstream processing, and, in the case of such compliance, sending a notification message to the user to submit the sample for downstream processing. Alternatively or additionally, the attributes of the sample used for evaluation include members selected from the group consisting of: size of the sample, dimensions of each patch of the sample, color of each patch of the sample, location of contours in each patch of the sample, and combinations thereof.

[0017] Optionally, the instructions establish a computer user interface configured to display to a user a different graphical representation of each of a plurality of portions of the sample, along with an identification of the portion, at least one attribute of the portion, and a location of the portion. Alternatively or additionally, the computer processing further comprises determining at least one attribute of the portion by z-axis analysis of an image of the sample. Alternatively or additionally, the computer user interface graphically indicates to the user a significant deviation in activity compared to activity associated with normal transfer of the sample. Alternatively or additionally, the computer user interface graphically indicates to the user that an attribute of the sample does not meet a minimum standard for downstream processing.

[0018] In another embodiment, a non-transitory storage medium is provided in which a data record associated with a catalogued sample is stored, the data record comprising: (i) an image of the sample, (ii) an image of identification data associated with the sample, wherein the image of the sample and the image of the identification data have been captured simultaneously, and (iii) a timestamp indicating the time when the image was captured. Optionally, the data record comprises a composite image, the composite image comprising the image of the sample and the image of the identification data. Alternatively or additionally, the image of the identification data is associated with a container in which the sample was initially stored, and the data record further comprises: (iv) an image of the sample transferred to a processing box, (v) an image of second identification data associated with the processing box, and (vi) a timestamp indicating the time when the image of the sample was transferred to the processing box and the image of the second identification data was captured. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The foregoing features of the embodiments will be more readily understood by reference to the following detailed description taken in conjunction with the accompanying drawings, in which:

[0020] Figure 1 is a perspective view of hardware and additional items of an apparatus for processing and cataloging tissue specimens according to an embodiment of the present invention;

[0021] Figure 2 FIG. 1 is a diagram showing a method for processing and cataloguing tissue specimens according to an embodiment of the present invention. Figure 1 A block diagram of the interconnection of the hardware components of the device;

[0022] Figure 3 According to an embodiment of the present invention, Figure 1 Block diagram of the software and hardware systems used in the device;

[0023] Figure 4 According to an embodiment of the present invention, a Figure 3 A block diagram of an overview of the processing performed by the components of the

[0024] Figure 5It is a diagram showing an embodiment of the present invention, which is used by technicians in Figure 3 A block diagram of a process performed in a system;

[0025] Figure 6 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of the processing performed by the imaging module;

[0026] Figure 7 is a diagram showing an embodiment of the present invention and Figure 3 A block diagram of processing associated with computer vision processing of an imaging module;

[0027] Figure 8 is a diagram showing an embodiment of the present invention and Figure 3 A block diagram of the processing associated with the z-axis analysis of the imaging module;

[0028] Fig. 9 is a diagram showing an embodiment of the present invention and Figure 3 A block diagram of the processing associated with the quality assurance function of the imaging module;

[0029] Fig.10 According to an embodiment of the present invention, Figure 3 The imaging module provides a representation of the display of the user interface;

[0030] Fig.11 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of the processing performed by the approval module;

[0031] Fig.12 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of additional processing performed by the approval module;

[0032] Fig.13 According to an embodiment of the present invention, Figure 3 The approval module provides a representation of a display of a user interface;

[0033] Fig.14 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of the processing performed by the management module;

[0034] Fig.15 is a diagram showing an embodiment of the present invention and Figure 3 A block diagram of additional processing associated with a management module;

[0035] Fig.16 According to an embodiment of the present invention, Figure 3 The management module provides a display representation of a user interface (UI);

[0036] Fig.17 is a representation of a display according to an embodiment of the present invention, which provides a method for removing a sample sheet from an intake container (initially in Figure 1 120) is transferred to a process box (initially in Figure 1 A user interface for the first step of FIG. 1 (shown as box 122); ···

[0037] Fig.18 is a representation of a display according to an embodiment of the present invention that provides a user interface for the second step of transferring a sample sheet from an ingestion container to a processing cartridge;

[0038] Fig.19 is a representation of a display according to an embodiment of the present invention that provides a user interface for a third step of transferring a sample sheet from an ingestion container to a processing cartridge;

[0039] Fig. 20 is a representation of a display according to an embodiment of the present invention that provides a user interface for the fourth step of transferring a sample sheet from an ingestion container to a processing cartridge. DETAILED DESCRIPTION

[0040] Definitions. As used in this specification and the appended claims, the following terms shall have the indicated meanings unless the context requires otherwise:

[0041] "Computer processing" is the performance of the described functions in a computer using computer hardware (e.g., a processor, field programmable gate array or other electronic combinational logic, or similar devices), which may operate under the control of software or firmware or any combination of these, or outside the control of any of the foregoing. All or part of the described functions may be performed by active or passive electronic components (e.g., transistors or resistors). When using the term "computer processing", we do not necessarily require the operation of a schedulable entity, or a computer program or a portion thereof, although in some embodiments, computer processing may be achieved by the operation of such a schedulable entity, or a computer program or a portion thereof. In addition, unless the context requires otherwise, "processing" may be achieved using more than one processor or more than one (single-processor or multi-processor) computer.

[0042] A "controller system" is a group of controllers.

[0043] A "group" includes at least one member.

[0044] A "specimen" is an object that is to be processed and catalogued for later reference. The term "specimen" includes, but is not limited to, tissue specimens that have been obtained for evaluation, and objects obtained from a crime scene for evidentiary or investigative purposes.

[0045] An "image" of a sample includes a photographic image as well as a video representation. The photographic image and the video representation may optionally be in a digital format, such as JPEG or MPEG format.

[0046] An "intake container" is a container used for initial storage of a patient tissue specimen and / or a container used for transport and presentation to the relevant pathology laboratory.

[0047] A "processing cassette" is a container with a slanted opening (or other similar features) that is used to process patient tissue specimens after receipt and ingestion in a pathology laboratory.

[0048] A "requisition" is a document that describes the source of a patient tissue specimen, the associated medical and patient identity, and the processing required for the tissue specimen. These typically accompany the patient tissue specimen intake container to confirm the patient's identity and provide details for processing in the laboratory.

[0049] A "tissue specimen" is biological material, including but not limited to tissue from an organ, dermis, or other area. Tissue may originate from medical procedures, including surgery, biopsy, animal testing, and autopsy, among other places.

[0050] A "touch screen" is a display device that allows a user to interact with a computer by touching areas on the screen.

[0051] A "software module" is a set of programs that causes a processor to perform a set of "computer processes" as defined herein.

[0052] An "electronic medical record" is a record stored in an electronic format that contains medical, biological, case, or test information related to a subject. Such records are sometimes referred to as (among other things) electronic medical records (EMRs), records in a laboratory information system (LIS), or records in a laboratory information management system (LIMS).

[0053] A "label" is a name applied to a container that includes data identifying the contents of the container. The name may be implemented in a variety of ways, including by ink applied directly to the container, by a printed label affixed to the container, and by engraving the container. The name may be in any of a variety of formats, including alphanumeric data, barcodes, QR codes, etc.

[0054] A “camera” is a device configured to capture images of a scene in two or three dimensions as a photograph or video representation, where the scene may include a sample.

[0055] The "frame" of an apparatus for handling and cataloguing samples is a structure mounted with a set of cameras configured to capture images of the samples, etc., and which also defines a working area for the samples to be imaged.

[0056] A "graphic selector" is a portion of a computer display that displays an icon, device, or button that is invoked by a mouse click or other graphical interaction method, and that is configured by a computer program to constitute a selection made by a user.

[0057] Figure 1 1 is a perspective view of the hardware of the apparatus for processing and cataloguing tissue specimens according to an embodiment of the present invention. The hardware components work independently and in combination with additional items for analysis of counts, sizes and other qualitative and quantitative visual features of tissue specimens.

[0058] The system hardware is housed in a main system frame 114 made of rigid plastic, aluminum or other suitable material; this embodiment utilizes a rigid plastic housing manufactured using a typical injection molding process. The frame is molded in three parts, which are then attached using molded fastener holes for partial alignment. The frame 114 includes support and mounting for all hardware components, including cameras, lighting, visualization screens, and processing components. One of these hardware components is the main camera 104 mounted within the main frame. The frame 114 also includes an internal platform 108 that allows environmental control of specimen evaluation (involving control of parameters such as light and temperature). The platform may include an area for placing tissue specimens and other materials thereon to use the system for the specimens and other materials. An intake container 120 carrying the specimen being evaluated is shown on the internal platform 108. A label 118 for the intake container 120 is also shown, which may include information identifying the sample. In this embodiment, the intake container is a Thermo Scientific brand Samco Histology Bio-Tite non-sterile specimen container. These items are available from Thermo Fisher Scientific, Inc., Waltham, Mass. The labels 118 shown can be produced by a common inkjet printer or more specialized laboratory processing equipment. A processing cartridge 122 can be used to process the specimen; during use of this embodiment of the invention, the specimen is transferred from the ingestion container to the processing cartridge, such as Figure 5 In this embodiment of the invention, the processing box is a Thermo Fisher brand Shandon TMBiopsy Processing / Embedding Cassette II; these items are available from Thermo Fisher Scientific, Inc., Waltham, Massachusetts. This embodiment of the system frame may also include multiple markings and other features within the system frame that are located within the field of view of the various cameras to allow for various calibration activities. This embodiment of the invention is designed to operate in an existing benchtop area and to address the risk of operator exposure to formalin or other fixatives, and is designed to operate in a fume hood or other area with air filtration. Other embodiments may include self-contained air filtration or air removal systems for safe operation in the presence of formaldehyde or other toxic materials.

[0059] The interior lighting system 102 can be used to control lighting conditions for computer vision and other activities. In this embodiment, a 12v DC power supply is used to power the LED lighting together with a voltage controller to control the voltage of the lighting and thereby control the light intensity. Optionally, as combined with Figure 3 As described, the intensity can be controlled manually or placed under the control of the computer vision system and processing module. The electrical and control wiring for such lighting can pass through the system processor cavity area 110. The various wiring can be connected to other components of the embodiment via a set of ports located at the rear of the main system frame 114 outside.

[0060] This embodiment includes multiple cameras that provide a visual record of the view of the area inside the main system frame and outside the frame. These cameras can be configured to operate at various resolution levels and focal length ranges depending on the environment. In this embodiment of the present invention, three cameras, namely the main camera 104, and auxiliary cameras 106 and 112, are used to provide a visual record of tissue specimens and containers, any labels on the containers, bar codes, and marks on the containers, and other documents (including but not limited to requests). The main camera 104 can have greater capabilities than the auxiliary cameras 106 and 112; for example, in this embodiment, the main camera 104 is a stereo camera capable of recording the height and configuration of the tissue specimen. In this embodiment, the first auxiliary camera 106 points to the inside of the system frame, and its field of view includes tissue specimens, intake containers, processing boxes, requests, and / or other documents associated with tissue specimens. The second auxiliary camera 112 points to a space close to the main system frame 114 but outside the main system frame 114, and its field of view includes a preparation area 116, in which associated documents, requests, and additional containers and boxes can be placed and visually recorded.

[0061] The main camera 104, which is a stereo camera, has a field of view directed toward a tissue specimen located within the system frame for quantitative and qualitative evaluation and analysis of the tissue specimen. In this embodiment, the main camera is a DUO-MINILX-LV1 available from Code Laboratories, Inc. of Henderson, Nevada. The camera has a resolution of 0.36MP (752×480) and a frame rate of 45FPS at this resolution. In this embodiment, the DUO-MINILX-LV1 can capture images using its two lenses and sensors, which can be used to determine the z-axis dimension of the tissue specimen being evaluated, such as in combination with Figure 8 All cameras are connected to computer hardware that performs the associated computer processing; in this embodiment, the computer's Universal Serial Bus (USB) 3.0 port is connected to a USB hub (such as a combination of a USB hub) as needed via a cable that passes through the main system frame 114 and the system processor cavity area 110. Figure 2 ) The outside of the main system frame is provided with a set of USB ports, which are connected to the cables from the computer.

[0062] In addition, as described, embodiments may include one or more auxiliary cameras 106 and 112. In this embodiment, two C920 video cameras available from Logitech of Newark, California are used. These cameras have a resolution of 1080p and capture 30 frames per second at this resolution. These cameras also connect to the computer hardware using a USB connection in a manner similar to the main camera 104.

[0063] The embodiment includes a touch screen 100 for a user interface for operation of the system. Other display and user interface arrangements are possible. Here, an Adafruit HDMI 7" 800×480 Display Backpack with a touch screen (available from Adafruit of New York, NY) was used. The display provides a resolution of 800×480 pixels and measures 7" diagonally. The display is located on the upper, outer side of the main system frame. The operator interacts with the display to manage visual recording and analysis of tissue specimens and other information associated with the processing and evaluation of the tissue specimens. The display also provides an interface for the operator to manage information and activities associated with other components of the embodiment, including but not limited to an approval module, a management module, and information provided from an electronic medical record system, such as in conjunction with a Figure 3 As described.

[0064] Figure 2 FIG. 1 is a diagram showing a method for processing and cataloguing tissue specimens according to an embodiment of the present invention. Figure 1206. In this embodiment, three cameras, one labeled "primary camera" 202 and two labeled "auxiliary cameras" 204 and 206, are connected to a USB hub 208 using standard USB connection cables. These USB cables power the cameras and carry image and other data. The USB hub is powered by a 12v DC power supply 210, which is provided by a standard AC / DC converter using 120v AC. The USB hub is also connected to a CPU (central processing unit) 212 via a standard USB cable. The CPU performs various computer processing, including software components for imaging analysis (imaging module 214 (in Figure 3 )). In this embodiment, the CPU is a common desktop computer running the Windows 10 operating system. The computer can be manufactured by multiple vendors, such as the Dell Inspiron Desktop Computer 3668 (available from Dell in Round Rock, Texas). The computer is provided with a keyboard and a display, the keyboard is from multiple vendors, including the Logtech K360 Wireless Keyboard (available from Logtech in Newark, California), and the display is from multiple vendors, including the Acer K222HQL 21.5LED (available from Acer in San Jose, California) monitor. In this embodiment, the computer is used for operation of the software components of the embodiment to operate the camera and other hardware components of the embodiment. The user can operate the camera via the monitor and keyboard described herein or via a combination of Figure 1 Describes the touch screen for interacting with the software.

[0065] Figure 2 The USB hub shown in FIG. 1 is also connected to the brightness controller 216 for combining Figure 1 The LED lighting system described above is an LED lighting system described above. In this embodiment, a 92-342 LED lighting dimmer from Calrad (Brightness Adjustment, available from Calrad in Los Angeles, California) is used to control a Philmore 4-LED 12VDC Square Light Module (Light Array, available from Philmore in Naples, Florida) 218. This lighting can be used to create a preferred lighting environment for image capture of tissue specimens, labels, and paper items placed within the main system frame. In this embodiment of the invention, an additional power supply 210 suitable for the LED lighting requirements is used.

[0066] Figure 3 According to an embodiment of the present invention, Figure 1 A block diagram of the software and hardware systems used in the device. Figure 3As shown in the figure, data related to the present invention is exchanged between three types of components. The first type is the VPB (VistaPath Biosystems) software 302; in this embodiment, there are three main software modules: imaging module 308, management module 310 and approval module 312, as combined Figure 4 The second type of component is the data hub 304, which is used for data storage and management and various computer processing. These data hubs retain image data, system settings, sample reports and other relevant information. In this embodiment of the invention, a data server is used for this function, which includes computer hardware for performing computer processing and data storage, such as a Dell PowerEdge T30 (available from Dell Inc. of Round Rock, Texas). The third type of component is VPB (VistaPath Biosystems) hardware 306. This type of component of the system is combined with Figure 1 Describes the physical hardware.

[0067] In this embodiment of the invention, the VPB hardware component delivers image data (described herein as a "video feed") to the imaging module. The Digital Vision Assistant or DVA 314 is a combination of Figure 1 The imaging module 308 sends data to control various aspects of the DVA function (described herein as "camera control"). The imaging module 308 operates on a computer located near the DVA hardware. The computer is connected to a local computer network via a hardwired connection such as an Ethernet cable, such as in conjunction with Figure 2As described. The imaging module sends the image data and specimen data, as well as other data, to the on-site data server 316. In this embodiment of the invention, the on-site data server is implemented by a computer system for storage, management, and delivery of data, including standard database properties, among other features. The on-site data server can directly access the electronic medical record 315, which is a database that stores patient information or other information related to the specimen being evaluated. Electronic record systems typically utilize software provided by companies such as Epic and Cerner. The on-site data server 316 can also directly access the VPB data server 318. The VPB data server can be located at a location far away from the site where the specimen is processed and cataloged; access to the data hub can be achieved via an Internet provider or by other common means. In this embodiment of the invention, the VPB data server represents a system for storage, management, and delivery of data, including standard database properties, among other features. The VPB data hub can contain data similar to that of the on-site data server and allows connection to additional VPB software. In the event of a communication failure between the on-site operation and the remote data server, both data servers 316 and 318 are able to work independently and maintain the ability to support the connected software components. In this embodiment, the VPB data hub is directly connected to the VPB software management module 310 and the approval module 312. For the interface between the VPB data server and the management module, the server sends user settings and other operational data to the management module, and receives data input and other user settings from the user, as well as other potential data exchanges. For the interface between the VPB data server and the approval module, the server sends image data, patient data, and other specimen data to the approval module, and receives data input from the user, including report approval or other report evaluation data, as well as other potential data exchanges. Figure 4 Describe these interfaces.

[0068] Figure 4 According to an embodiment of the present invention, a Figure 3 A block diagram of an overview of the processing performed by the components of FIG. Figure 3Each of the five components of is a separate dedicated column, and the logical flow of these components and their direct data connections is shown. When the user starts the specimen evaluation 410, the imaging module function 400 begins. The imaging module then controls the camera and lighting operations 412, and can, for example, set the camera aperture and lighting intensity and other operations. The image data from these cameras is then analyzed in the computer vision pipeline of the imaging module 414 (in this description of the embodiment of the present invention, in the context of computer vision processing, the pipeline refers to a collection of software processes that produce certain image analysis). One component of the computer vision pipeline identifies the tissue specimen, calculates the boundaries of the specimen, and generates the size of each specimen in 3 dimensions (x, y, z). The computer vision pipeline can also calculate additional quantitative information about the tissue specimen, such as color and count. Another component of the computer vision pipeline within the imaging module may use object feature recognition (OCR) to identify various tags and barcodes associated with the specimen being evaluated. This data is evaluated 416 in a quality control process where all relevant data is checked; these processes determine whether text, barcodes, QR codes, and other encoded text information are internally consistent and associated with the correct case or other specimen identifier. Fig. 9 This process is further described. Although in the current embodiment, the quality control process 416 is performed only once, it can alternatively be performed repeatedly throughout the evaluation and processing of the specimen. Finally, in process 418, the data generated by the imaging module is aggregated and sent to the on-site data server 402, which data includes but is not limited to the camera image, the size and color of the tissue specimen, and patient information.

[0069] For this embodiment of the present invention, field data server 402 stores and processes data 420 from DVA and imaging module, connected to described component using local computer network.In field data server, case and other specimen data provided from imaging module are compared with electronic patient record 424 in process 422, which is the extension of quality control process 416.Related texts previously marked on container and paper in the visual field of imaging hardware (camera) are evaluated to confirm that they are internally consistent and associated with correct case or other specimen identifier.Evaluation of this information is performed via data exchanged by field data server and electronic patient record 424.These and other related data are also stored in field data server for access by other software modules.

[0070] Patient and specimen information stored on the field data server 402 is also sent to the VPB data server 404. It is noteworthy that in this embodiment, redundant data may exist on the field data server 402 and the VPB (Vistapath Biosystems) data server 404. This allows independent operation of the various components of this embodiment of the invention in the event of a communication failure between the field data server and the VPB data server. In process 428, while the connection is active, the redundant data is compared at different points, with conflict management operating according to common database system protocols. For a given case or specimen evaluation, when all required data is stored and properly evaluated, the process terminates 430. The approval module 406 and the management module 408 are each directly connected to the VPB server 404, although in other embodiments there may be additional data connections directly connected to the field data server 402.

[0071] The approval module 406 receives and aggregates data such as patient identity, image data, analysis of the data from the computer vision pipeline, and other relevant data 432. Based on the criteria entered by the user for the operation of the specimen evaluation process, the module can create a priority for the specimens to be evaluated by the user. The development of the priority queue 434 can include criteria such as tissue type, time of entry into the system, and requirements for processing, as well as other criteria. In addition, the approval module can aggregate data from the VPB data server to generate a specimen description 436, which is referred to as a general report in this embodiment of the present invention, for the technician to evaluate. The general report includes a written description of the (multiple) specimens, which includes a count of the number of slices, the size of each slice, and a description of the color and possible other characteristics. In other embodiments, the descriptive report can be identified with another term and can include additional or different attributes for the specimen being evaluated. The approval module can also deliver the report and associated data to the user for their evaluation. In this embodiment, the user interface displaying the general report can include the text of the generated report, patient information, and an indication that all related labels and paperwork are associated with the expected specimen. Combination Fig.10 An example of such a user interface for this embodiment is shown. After the user evaluates the information, the user may accept the report, reject the report, or request a modification of the specimen evaluation or report. The technician's approval or rejection of the report is recorded in process 438. Fig.11 The details of these processes are described in more detail in .

[0072] The management module 408 aggregates operational and image data 440 from the VPB data servers and performs statistical analysis of the information to evaluate various aspects of the processing and the specimen. In this embodiment, the analysis includes, for example, tracking and statistical analysis 442 of each processing step, the operation rates and error rates of individual users, details of the processing performed, and any categorical information about the case and / or laboratory operation. Various presentations of this and other information may be provided to the user through various user interfaces. Any user preferences and / or changes to user-generated changes to the management module are recorded for subsequent use 444. In conjunction with Fig.13 and Fig.14 These software operations are further described.

[0073] Figure 5 It is a diagram showing an embodiment of the present invention, which is used by technicians in Figure 3 . In this embodiment, the user performs two separate processes, tissue processing 502 and general report review 504, shown in the first and second columns of the figure, respectively. For tissue processing 502, the user begins the evaluation of the specimen 506 by placing the material in the preparation area and preparation area 508 of the current embodiment of the present invention. If the specimen is in a closed ingestion container, the user will remove the top of the container to allow the system to review 510. The user will then be presented with a display of the following: an image of the specimen being evaluated (via a user interface or UI), as well as a report describing the specimen and verification that all materials (including labels and paperwork) are associated with the correct patient record. The user will evaluate the acceptability of this information 512. If the user finds that the information presented is unacceptable, the system will provide the technician with the option of adjusting and modifying the information and image 514. This may include, but is not limited to, modifications to the lighting and computer vision analysis of the specimen and modifications to the descriptive report, as well as other possibilities. If the technician deems the specimen processing to be acceptable 512, the user will transfer the specimen from the ingestion container to the processing box 516. In combination Fig.17 This processing is further described.The user then removes the specimen and associated documents from the DVA 518. Additional processing of the specimen then occurs 520, outside the scope of this embodiment of the invention.

[0074] At this point, this embodiment of the invention will also notify the technician 522 that the specimen (case) is available for review and potentially for approval using the general report review component 504. The technician can select one of the cases available for review from the list (queue) 524; the technician will then review the images and other data for the case 526. If the case report and associated information are acceptable to the user (technician) 530, they will use the provided UI to record their approval and move the case to a separate queue for downstream processing 532. If the case report and associated information are unacceptable, the user will follow the provided prompts associated with the tissue processing workflow 502. If a new image of the specimen is required 534, the user of this process will be guided 536 through options for modifying and / or correcting the image capture. If a new image is not required, the user will be guided 538 through various options for modifying and / or correcting additional information associated with the specimen. The corrected specimen processing information will be re-evaluated as part of the tissue processing workflow 512 and then re-sent to the queue for general report review processing 504.

[0075] Figure 6 According to an embodiment of the present invention, Figure 3 606 , and passes these images to the computer vision pipeline. The computer vision analysis identifies and generates a description of the individual specimens 610 , combined with the Figure 7 The z-axis calculation process determines the z-axis dimension (height) of the specimen via stereo sensing 612, combined with Figure 8which is further described. The data from the image analysis is then loaded into the user interface 614 and 616. The user then views the image and text showing the analysis performed by the computer vision pipeline, including but not limited to the size, count, and color of the specimen. The user then evaluates whether the image and data are acceptable 618. If the image and associated analysis are unacceptable, the user adjusts the computer vision settings and / or the input materials (specimens, documents, etc.) 620. The imaging module then modifies the system settings to match these modifications 622, and the system instructs the camera to capture images for subsequent analysis. If the information provided in 614 and 616 is acceptable (yes), the quality assurance process analyzes the text generated from the images of the materials (e.g., documents, processing boxes, ingestion containers, and any labels affixed to those materials) to determine whether all such information is internally consistent (i.e., the specimens and containers used are associated with the correct patients and patient electronic records) 624. If all relevant data is consistent, it is aggregated and passed to the DVA image and case storage database 626 (which is a storage system that is compatible with the DVA image and case storage database). Figure 4 If the data is not internally consistent, the user must modify the associated container, label, and / or document 628, and the imaging module will then instruct the camera to capture a new image of the specimen and associated container 606. Fig. 9 These quality assurance processes are described in more detail in .

[0076] Figure 7 is a diagram showing an embodiment of the present invention and Figure 3 A block diagram of the processing associated with the computer vision processing of the imaging module of FIG. The computer vision analysis identifies and generates a description of a specimen imaged from a device camera (initially in Figure 6 610 of the process). Processed by the imaging module, the device camera images are sent to these computer vision processes 702. The two images from the auxiliary cameras may contain text-containing components such as applications, boxes, and various labels affixed to these materials, and are sent to an optical character recognition (OCR) engine to extract meaningful text; this is processed for both auxiliary camera A 704 and auxiliary camera B 706. For example, using OpenCV's text recognition capabilities, text is identified from the provided images and translated into text fields (patient name, date of birth, etc.), which is processed for both auxiliary camera A 708 and auxiliary camera B 710. This data is then aggregated in process 712 and then presented to the user for their evaluation 714 (initially combined with Figure 6 618 description of processing).

[0077] In addition to the processing described in this figure, the image from the main camera, which may include an image of a vial containing a tissue specimen, is also sent to a CV (computer vision) pipeline 716. Through a series of processes, the CV pipeline identifies or segments the tissue specimen from the background of the image. First, in this embodiment of the invention, a Hough transform is used to separate the ingestion container location from the rest of the image 718. The image is then cropped to show only the ingestion container 720. The cropped image is processed to identify the subset of the image (called a threshold) where the specimen is located 722. In this embodiment, a two-level threshold is used, but other methods are possible. The extracted contours can be used to calculate the x and y dimensions of the tissue specimen using a pixel-to-millimeter conversion suitable for this embodiment of the invention 724. The color of the specimen is determined by averaging the RGB values ​​within the extracted contours and comparing them to a series of manually defined color bins 724. The extracted contours are then sent for additional processing to determine the z-axis dimension (vertical dimension) 726 (in combination with Figure 8 The data generated by these processes (including but not limited to specimen outline location, size, and color) are then aggregated 712 for user evaluation 714, such as initially combined Figure 6 The processing described in 618.

[0078] Figure 8 is a diagram showing an embodiment of the present invention and Figure 3 In particular, the figure also explains the processing required to determine the z-axis dimensions of a tissue specimen identified by a segmentation process via stereo sensing, such as in combination with Figure 7 The series of processes begins 802 when the outlines and positions of the tissue specimen identified within the image provided by the device are received. These outlines and positions can also be described as regions of interest (ROIs) of the image, as is done here. Figure 1As noted, for this process, a DUO-MINILX-LV1 stereo camera was used, which includes two cameras ("Camera 1" and "Camera 2") set at a known distance from each other. The ROI data, i.e., the i-th sample ROI, is sent to the first of the two stereo cameras for analysis 804. The location of a feature point representing the maximum possible height point of the specimen is identified from the stereo image of the first camera within the stereo camera setup 806, in this case the centroid of the ROI of the i-th tissue specimen. The same feature point is then identified from the stereo image of the second camera within the stereo camera setup by finding the maximum value of the normalized cross-correlation of the feature point with the image from the second camera 808. Next, a similar triangle is used to identify the z-axis distance from the stereo camera to the feature point 810. Assuming that the z-axis distance from the stereo camera to the bottom of the vial is known, and assuming that the tissue specimen will be placed at the bottom of the vial, the z-axis dimension of the i-th specimen is calculated 812. If there are additional sample outlines for analysis 814, the process is repeated until the z-dimensions of all identified tissue specimens are found 816. All relevant data is then saved 818 and aggregated 820 with other specimen data (including x-axis and y-axis dimensions, color of tissue specimens, text parsed by the OCR engine, and the raw image from the camera) and passed to the combined Fig. 9 Describe the quality control process.

[0079] Fig. 9 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of the processes associated with the quality assurance functions of the imaging module. These components take as input the aggregated data of the image accumulated by the imaging module and the relevant data evaluated by the user, as initially combined Figure 6 Furthermore, the process described in this figure also describes in detail the evaluation of the initial combination Figure 6 The "correct specimen and container to be used" process (here represented as process 906) described in process 624 of the present invention. Using these images deemed acceptable by the user, the quality assurance function compares the text fields parsed from the processed images of the cameras of the present invention, including information such as patient name and date of birth 908. If the text data collected from these cameras is inconsistent, the system will lock the user interface (UI) to prevent further specimen processing 910 and prompt the user to take corrective action 912. The user can resolve the inconsistency and Figure 6The system boot process restarts the imaging module. It is noteworthy that all of these processes occur within the imaging module 902. If the resulting text is internally consistent, the aggregated data is passed to the DVA image and case repository within the on-site data server 904. The data is then compared with the case information stored in the user's electronic record system based on the same criteria 914. The internal consistency of the data is then evaluated again 916. If the relevant text and additional data are internally consistent, the data is sent to the DVA image and case storage database and the evaluation of the data consistency is recorded 918. If the data is not internally consistent, the user interface (UI) is locked again 910 and the user is prompted to correct the conflict 912. Through all of these processes, the internal consistency of the checked data is communicated to the user by displaying a check mark symbol on the relevant user interface (UI). Although the quality assurance process described here is performed once at the end of the imaging module, in other embodiments of the present invention, it can be performed multiple times throughout the specimen processing steps.

[0080] Fig.10 According to an embodiment of the present invention, Figure 3 The imaging module 308 provides a display representation of a user interface (UI). The interface allows a user to evaluate the output from specimen imaging and report generation to assess the accuracy of the specimen evaluation process. Fig.11 As described, final approval of specimen processing can be performed in a separate step. In this embodiment, this separation of user actions allows users with lower levels of training and certification to process specimens and evaluate the success of the processing, while final approval of the overall report can be performed by a technician with a higher level of certification without being placed with the specimen(s). Case information 1004 including patient name, medical record number (MRN), and date of birth (DOB) is displayed for user review. User identification 1010 (in this embodiment, user name and time of use) is displayed to confirm that the correct user is associated with specimen processing. The user can modify viewing settings and other options by pressing a graphical selector indicated by a "gear" icon 1008.

[0081] The user interface displays multiple images with various associated information. Process 1002 is an image of the specimen(s) for a given case. A description 1006 of the specimen (also referred to as a "general report" in this embodiment of the invention) is presented for the user to review. This text generated from the computer vision pipeline should accurately describe the specimen represented in image 1002; this is where the user can evaluate the information and choose to submit the report for approval, or return to other processes to modify the image or related information (such as the original combined Figure 6In this embodiment, if the user wants to display the descriptive report in a different template and / or format, this type of change can be made by selecting various graphic tabs 1014. In order to show the user all containers, labels and other text information associated with the current case, the user interface displays an image and marks the image 1012 with a check mark icon to indicate that the data being evaluated is internally consistent, such as in combination with the Fig. 9 As described. In this embodiment, the evaluation box is processed for inclusion; in other embodiments, paper and other labels associated with a given case will be included in the evaluation. If the information presented is acceptable, the user can indicate this decision by pressing the "Submit Report" graphic selector 1016. If the information is unacceptable, the user can press the "Return" graphic selector 1018. The user will then be presented with a series of options to modify the current report, such as Figure 6 Initially described.

[0082] Fig.11 is a diagram showing an embodiment of the present invention. Figure 3 A block diagram of the processing performed by the approval module of Figure 4 The approval module allows for review, modification, and potential approval of specimen evaluations, as described in process 406. The following is a more detailed description of the logic flow for queue generation for this module.

[0083] The user will initiate the approval module process via the user interface 1104. The module will then access the current priority criteria for the active user 1106. The system will then query the DVA case and image repository on the VPB data server (initially combined with Figure 4 Additional priority criteria (initially combined with Figure 4 Based on this data, an appropriate queue will be developed for the user 1112. The development of the priority queue may include criteria such as specimen type, time of entry into the laboratory system, and requirements for processing, among other criteria. One of a number of common queue algorithms may be applied here to develop the prioritization. The queue is then presented to the user 1114 using a user interface, after which the user may select cases for review 1116. The selected cases are then sent to the approval module for additional processing to generate a summary report 1102, as combined with the Fig.12 As described.

[0084] Fig.12 is a diagram showing an embodiment of the present invention. Figure 3In particular, the figure depicts the generation and review of a summary report 1200 and the presentation of the report to a user for evaluation and possible approval 1202. Once a case is selected from a case queue (e.g., Fig.11 1116 and shown again here for continuity at 1204), the process of generating the total report begins. Query the VPB data server (initially in conjunction with Figure 4 404) to pull image and text data for the selected case 1206. Then, a user-defined template for the overall report is pulled 1210. The template may include the type and location of data for display; preferences for the template may be entered by the user through appropriate approvals 1208 and stored in the VPB data server. The approval module will then generate an appropriate UI that displays the case data 1212 for user inspection 1218. If the user accepts the case data presented, the VPB data server will be queried for a database row that matches the case 1214. In addition, the overall report text displayed to the user will be entered into the database location and the status of the report will be changed to "Approved" 1216.

[0085] If the user does not approve the master report generated by the approval module 1218, the user will be guided through options to modify or reject the report, as described in the graphical column "Manage User Responses" 1202. The user will be prompted to decide whether the master report can be modified to make the report acceptable 1220. If no modification is possible to allow approval, the user will enter criteria for the failure and their response will be stored in the VPB data server 1222. Such criteria may include, for example, issues related to image quality or image interpretation performed by the imaging module. If modifications to the master report can allow approval of the report, the approval module will use the UI to propose categories for modifications to the report 1224. These categories may include the layout of the report, how the size of the tissue is presented, or other possible modifications. The user will be presented with a UI to make such modifications to the report 1226. The revisions to the master report will be sent to the VPB data server for storage 1230. Revisions to the master report will then be generated 1228 and presented to the user for them to determine whether the report is acceptable 1218. Based on the user response, the approval module will again prompt the user through the approval process as described above.

[0086] Fig.13 According to an embodiment of the present invention, Figure 3The approval module provides a representation of a display of a user interface (UI). The interface allows a user to evaluate a specimen (total) report and associated data. In this embodiment of the invention, the UI is used by a qualified user to provide final approval of the image and text reports associated with the specimen. Case information such as patient name, medical record number (MRN), and date of birth (DOB) are displayed for user review 1304. A user identification 1310 (in this case, the user's name and time of use) is displayed to confirm that the correct user is associated with the specimen report review. The user can view and modify settings and other options by pressing a graphical selector indicated by a "gear" icon 1309.

[0087] Fig.13 The user interface in displays multiple images with various associated information. Process 1302 shows images of (multiple) specimens for a given case. A description of the specimens (overall report) 1308 is also shown. The information generated by the imaging module is a textual description of the specimen represented in the (multiple) image 1302; this is one of multiple criteria by which the user can evaluate the output and choose to confirm or mark the information. If the user wants to display the descriptive report in a different template and / or format, this type of change can be made by selecting one of the appropriate tabs 1314. In order to show the user all of the container labels and other textual information associated with the current case, the user interface displays the two images and marks them with a check mark icon to indicate inclusion in the correct case, as initially combined. Figure 4 In this embodiment, the UI shows the ingestion container 1306 and the processing box 1312, and displays a check mark icon if the information is internally consistent. If the information presented in the UI is acceptable, the user can indicate this decision by clicking on the "Confirm Total Report" graphic selector 1316. If the information is not acceptable, the user can click on the "Mark for Review" graphic selector 1318. The user will then be presented with a series of options to describe the problems associated with the current report, initially in conjunction with Fig.12 Described.

[0088] Fig.14 is a diagram showing an embodiment of the present invention. Figure 3 14. The diagram is a block diagram of the processing performed by the management module of FIG. 14. The management module aggregates operational and image data to review various aspects of the processing and informs the user of relevant information about the specimen. In particular, the diagram describes the processing for management module startup 1400, and completed analysis 1402 of a case cohort.

[0089] The management module is started by the user 1404, who then logs into the system 1406. The module pulls permissions and user profile 1408 for a given login, and displays an initial UI (e.g., Fig.16 The user selects the information category of interest for analysis 1412. The module then queries the VPB data server to pull relevant data for analysis 1414. This data can be used for queue completion 1402 as described herein, as well as in conjunction with Fig.15 Additional analysis described 1424. To analyze the percentage of the queue that has been completed, the module pulls the data for the entire queue and the data for the completed cases 1416. The percentage of pending cases for each technician as a percentage of the total queue for that technician is then calculated 1418. The percentage of pending cases for a given group as a percentage of the total queue for that group is then calculated 1420. The group can be defined, for example, as a process step or a given laboratory. The module then displays these analyses to the user 1422. In other embodiments of the invention, a representation of the analysis can be printed or otherwise stored for later review.

[0090] Fig.15 is a diagram showing an embodiment of the present invention. Figure 3 The management module aggregates operational and image data for use in evaluating various aspects of the processing and notifies the user of relevant information about the specimen. In particular, the figure describes additional analysis provided to the user for them to evaluate the performance of the operation. Fig.14 As described, relevant data is pulled from the VPB data server for combination Fig.14 The analysis described and the analysis described herein in process 1507. To analyze case turnaround time (abbreviated herein as TAT) 1502, the processing time is averaged for each user 1508 and for all users 1510. These averages are then displayed for user review 1512. In order for the user to evaluate case capacity for a given time period 1504, the data obtained from process 1507 is used to calculate a total case capacity for a default time period 1514. This calculated data is displayed for user review 1516. The user can then select a different time / date range for calculating the total case capacity 1518. The module will recalculate the totals based on the new time range 1520 and present the analysis for user review 1522. In other embodiments of the invention, various graphics and other visualization tools can be generated to support user review of this and other information.

[0091] An analysis of errors or other quality issues is generated through the processing described in the QA (quality assurance) question column 1506. Fig.14As indicated, relevant data is pulled from the VPB data server for combination Fig.14 The analysis described above and the analysis described here (process 1507). For this analysis, the error rate will be used to calculate the total errors and average errors 1524. These average errors and total errors can be filtered by type (e.g., error type) 1526; the analysis is displayed for user review using a default time period 1528. The user can then select a different time period for analysis 1530, and the module will filter the relevant data 1532. The user can then select a report type to summarize the desired QA analysis for presentation 1534. The module will then filter and aggregate the relevant data 1536 and display the report for user review 1538. In other embodiments of the present invention, a representation of the analysis can be printed or otherwise stored for later review.

[0092] Fig.16 According to an embodiment of the present invention, Figure 3 The management module provides a display representation of a user interface (UI). The user interface allows a user to evaluate the initial combination Fig.14 and Fig.15 The interface can display various work queues, case completions (totals and rates), and other metrics generated by the management module. The user can select from these information types via a graphic selector 1604. In this example, the diagram illustrates the use of a "metrics" selector. Additionally, various settings can be accessed by pressing a "gear" icon 1606. A user identifier 1608 (user name and usage time in this embodiment) is shown to confirm the user's identity based on their initial login information, which determines the data that can be accessed for review. The interface is shown in combination with Fig.14 and Fig.15 Various operational data and other data processed by the management module described are shown. Table 1610 is shown, which shows the percentage of specimens completed for a given user queue. The average number of specimens evaluated in a 24-hour period for each user is shown in additional table 1612. The volume of specimens processed in a given time range is also shown as a bar graph 1614; the date range and other criteria for filtering this data are shown here for the user to select. Additional chart 1616 indicates the percentage or count of errors over the selected time period. This QA analysis (in conjunction with Fig.15 Description) can be presented for each technician, for the laboratory as a whole, or for a given processing step where the error occurred. The user can return to other processing, such as single specimen evaluation or approval, by pressing the start case graphic selector 1602.

[0093] Fig.17is a representation of a display according to an embodiment of the present invention, which provides a method for removing a sample sheet from an intake container (initially in Figure 1 120) is transferred to a process box (initially in Figure 1 The process described here is first combined with Figure 5 Extension of the process described for (tissue) sample transfer. The sample identification data, in this example the patient's name 1700 and date of birth 1702, are located in Fig.17 1704 of the display. A top view image 1704 of the ingestion container is shown here; in this step, the container has the screw top in place. In addition to this top view image of the ingestion container, an image 1710 of any label (including a barcode, etc.) affixed to the ingestion container is also shown. A top view image 1706 of the processing cartridge is also shown. Fig.17 The image also includes any identifying text and / or barcode 1708. Fig. 9 As described, this embodiment of the invention compares the identification data imaged herein from the ingestion container and the processing cartridge and compares this information internally and with the case and specimen information stored in the LIS to assess internal consistency. If it is determined that information items 1708 and 1710 are consistent with each other and with the data in the LIS, a check mark symbol 1712 is displayed on the user interface. If one or more of the identification information sets are inconsistent, an x ​​mark symbol is displayed and the user is prompted to make correction(s) to the settings before continuing, such as in conjunction with Fig. 9 When the identification information is consistent, the user can continue to perform additional steps to transfer the sample sheet from the intake container to the processing box, such as in combination with Fig.18 As described.

[0094] Fig.18 is a representation of a display according to an embodiment of the present invention, which provides a user interface for the second step of transferring the sample sheet from the uptake container to the processing box. Fig.17 As depicted, when the check mark symbol 1712 is displayed on the user interface, the user can continue the transfer of the sample piece by specifically removing the top of the intake container shown in image 1704. Fig.18 In the example, the top of the spiral has been removed and the interior of the ingestion container can be seen in image 1704; the computer vision component of this embodiment of the present invention will then process the top-down image 1704. In this example, two samples 1802 and 1804 are identified by the computer vision component. As identified by this embodiment of the present invention, these samples are displayed as thumbnail images 1806. In addition, these sample images 1802 and 1804 are combined Figure 6The computer vision component described above performs the analysis and also displays the resulting dimensions of each sample slice 1806. This display 1806 of the sample slice image includes symbols that convey to the user the location of a given sample slice. When a sample slice is located in the ingestion container, a circle is displayed next to the sample slice image. If the user disagrees with the identification of the sample slice displayed, the user can select the sample slice directly on the user interface. An additional dialog window will be presented to allow modification of the sample slice identification, such as in conjunction with Figure 6 If the user agrees with the identification and location of the sample pieces displayed in the user interface, the individual sample pieces can be transferred to the processing box, such as in combination with Fig.19 As described.

[0095] Fig.19 is a representation of a display according to an embodiment of the present invention that provides a user interface for the third step of transferring a sample slice from an intake container to a processing box. In the third step of transferring the sample slice, the user has physically transferred the sample slice 1802 from the intake container to the processing box. Therefore, the image 1704 showing the intake container only shows the single sample slice 1804 remaining in the container. The image 1706 showing the processing box now shows the sample slice 1802 that the user has transferred from the intake container. In addition, the computer vision component of this embodiment has identified the presence of the tissue located in the processing box. Therefore, the display 1806 of the sample slice image now includes a square next to the thumbnail image of the transferred sample slice 1802 to indicate to the user that the slice has been transferred to the processing box. If the user disagrees with the identification and location of the displayed sample slice, the user can select the sample slice directly on the user interface. An additional dialog window will be presented to allow modification of the sample slice identification, such as in conjunction with Figure 6 If the user agrees with the identification and location of the sample pieces displayed in the user interface, additional sample pieces may be transferred to the processing box, such as in conjunction with Fig. 20 As described.

[0096] Fig. 20is a representation of a display according to an embodiment of the present invention that provides a user interface for a fourth step of transferring a sample sheet from an intake container to a processing box. In the fourth step of transferring the sample sheet, the user has physically transferred the second sample sheet 1804 from the intake container to the processing box. Therefore, the image 1704 showing the intake container shows that there are no visible sample sheets. The image showing the processing box 1706 now shows the two sample sheets 1802 and 1804 that are present because the user has now completed the transfer of the two sample sheets from the intake container. In addition, the display 1806 of the thumbnail sample sheet images now shows a square next to the two sample sheet thumbnail images 1802 and 1804 to indicate to the user that the computer vision component has identified the two sample sheets that are present in the processing box. When this transfer of the sample sheets is completed, if the computer vision component identifies the same sheet count for the sample sheet as the initial count of the sheet from the intake container, the user will be prompted to confirm that the process is complete via a new dialog box shown together with the user interface. If the computer vision component does not identify a chip count for a sample chip that is the same as the initial count of chips from the ingest container, the user will be prompted to locate any missing or modified sample chips via a new dialog box shown with the user interface.

[0097] The computer processing performed in the embodiments of the present invention may be implemented in many different forms, including but not limited to computer program logic for a processor (e.g., a microprocessor, a microcontroller, a digital signal processor, or a general purpose computer), programmable logic for a programmable logic device (e.g., a field programmable gate array (FPGA) or other PLD), discrete components, integrated circuits (e.g., an application specific integrated circuit (ASIC)), or any other device including any combination thereof.

[0098] The computer program logic that implements all or part of the functions previously described herein can be implemented in various forms, including but not limited to source code form, computer executable form, and various intermediate forms (e.g., forms generated by assemblers, compilers, networkers, or locators). The source code may include a series of computer program instructions implemented in any of various programming languages ​​(e.g., object code, assembly language, or high-level languages ​​such as Fortran, C, C++, JAVA, or HTML) for use in various operating systems or operating environments. The source code may define and use various data structures and communication messages. The source code may be in computer executable form (e.g., via an interpreter), or the source code may be converted (e.g., via a translator, assembler, or compiler) into a computer executable form.

[0099] A computer program may be permanently or temporarily fixed in any form (e.g., source code form, computer executable form, or intermediate form) in a tangible storage medium such as a semiconductor storage device (e.g., RAM, ROM, PROM, EEPROM, or flash programmable RAM), a magnetic storage device (e.g., a magnetic disk or fixed disk), an optical storage device (e.g., a CD-ROM), a PC card (e.g., a PCMCIA card), or other storage device. A computer program may be fixed in any form in a signal that can be transmitted to a computer using any of a variety of communication technologies, including but not limited to analog, digital, optical, wireless, network, and network-to-network interconnect technologies. A computer program may be distributed in any form as a removable storage medium accompanying a printed or electronic document (e.g., shrink-wrapped software or tape), preloaded with a computer system (e.g., on a system ROM or fixed disk), or distributed from a server or electronic bulletin board via a communication system (e.g., the Internet or the World Wide Web).

[0100] Hardware logic (including programmable logic for a programmable logic device) that implements all or part of the functionality previously described herein may be designed using conventional manual methods, or may be designed, captured, simulated, or electronically recorded using various tools such as computer-aided design (CAD), hardware description languages ​​(e.g., VHDL or AHDL), or PLD programming languages ​​(e.g., PALASM, ABEL, or CUPL).

[0101] Although the present invention has been particularly shown and described with reference to certain embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the appended clauses. Although some of these embodiments are described in the claims by processing steps, the present invention also includes an apparatus comprising a computer with an associated display capable of performing the processing steps in the following claims. Likewise, a computer program product comprising computer executable instructions for performing the processing steps in the following claims and stored on a computer readable medium is also included in the present invention.

[0102] The embodiments of the present invention described above are intended to be exemplary only; many variations and modifications will be apparent to those skilled in the art. All such variations and modifications are intended to be within the scope of the present invention as defined by any of the appended claims.

Claims

1. A device for processing and cataloguing samples in containers having labels with initial identification data, wherein: The sample is to be transferred to a processing box having second identification data thereon, and the device comprises: a frame defining a working area, wherein the working area includes a first position and a second position for receiving the container and the process cartridge, respectively; A camera assembly mounted on the frame; a controller system coupled to the camera head, the controller system comprising a controller and a non-transitory memory storing instructions, the instructions, when executed by the controller, causing a computer process to be performed, the computer process comprising: Enables cameras in the camera collection to capture: an image of the sample in the working area; an image of the initial identification data; an image of the second identification data; and a collection of images recording transfer of the sample from the container to the processing box; converting the initial identification data and the second identification data into a first digitally encoded text and a second digitally encoded text, respectively; and comparing the first digitally encoded text and the second digitally encoded text to ensure consistency between the initial identification data of the label of the container and the second identification data of the process cartridge; processing the image of the sample to generate a data output characterizing a property of the sample; and integrating the data output into a catalog record output characterizing the sample, the catalog record output also including the initial identification data; analyzing said set of captured images recording said transfer of said sample from said container to said processing cartridge to identify a transfer error; The transfer error includes a transfer error in which a part of the samples to be transferred has been marked as lost.

2. The device according to claim 1, wherein: Processing the image of the sample includes forming a composite image including the image of the initial identification data and the image of the sample.

3. The device according to claim 1, wherein: The catalog record output includes an image of the specimen, an image of the initial identification data, and a timestamp indicating the time at which the image of the initial identification data and the image of the specimen were captured.

4. The device according to claim 1, wherein: The computer processing also includes causing a selected one of the cameras to capture the set of images.

5. The device according to claim 4, wherein: The set of images captured is a video recording.

6. The device according to claim 1, wherein: The transfer error includes a transfer error in which at least a portion of the sample has not been transferred to the processing box.

7. The device according to claim 1, wherein: The transfer error includes a transfer error in which the portion of the sample to be transferred already remains in the container.

8. The device according to claim 1, wherein: The transfer error includes a transfer error in which the sample appears to have been transferred to an incorrect processing cartridge based on a failure of the second digitally coded text to match the first digitally coded text.

9. The device according to claim 1, wherein: The computer processing further includes evaluating the data output characterizing the properties of the sample to meet minimum criteria for downstream processing and, if so, issuing a notification message to a user to submit the sample for downstream processing.

10. The device according to claim 9, wherein: The attributes of the sample used for the evaluation include members selected from the group consisting of: size of the sample, dimensions of each patch of the sample, color of each patch of the sample, contour positions in each patch of the sample, and combinations thereof.

11. The device according to claim 1, wherein: The instructions establish a computer user interface configured to display to a user a different graphical representation of each of a plurality of portions of the sample, along with an identification of the portion, at least one property of the portion, and a location of the portion.

12. The device according to claim 11, wherein The computer processing also includes determining at least one property of the portion by z-axis analysis of the image of the sample.

13. The device according to claim 11, wherein: The computer user interface graphically indicates to the user a transfer error in which at least a portion of the sample has not been transferred to the processing cartridge.

14. The device according to claim 11, wherein: The computer user interface graphically indicates to the user that properties of the sample do not meet minimum standards for downstream processing.

Images