Automatic sample loading and unloading method and device for fully automatic AI microscopic analysis instrument
Through the machine vision module and gray decision model, efficient sample loading and unloading of fully automatic microscopic detection analyzers is achieved, solving the problems of low loading and unloading efficiency and poor stability in the prior art, and improving the accuracy and stability of loading and unloading.
Patent Information
- Application Number
- CN202411357671.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-27
AI Technical Summary
The sample loading and unloading process of existing fully automatic microscopy analyzers is inefficient and have poor stability, and are prone to errors.
The two-step method based on the machine vision module is used to collect sample container images, identify the autonomic features and spatial distribution features, combine the transmission control characteristics of the loading and unloading mechanism to build a gray decision model, determine the automatic loading and unloading strategy, and perform loading and unloading control through the intelligent central control system. Assisted gray predictive analysis for feedback adjustment.
Improve loading and unloading efficiency, improve loading and unloading accuracy and stability, and reduce operating errors.
Smart Images

Figure CN118883451B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a sample automatic loading and unloading method and device for a fully automatic AI microscopic analysis instrument. Background Art
[0002] Fully automatic microscopic analysis instruments have been widely used in medical and industrial testing. Their high efficiency and intelligence have improved the testing efficiency. The sample loading and unloading process in the prior art relies on manual or semi-automatic devices, which are difficult to operate and prone to errors in the processing of a large number of samples. There are technical problems such as low loading and unloading efficiency and poor loading and unloading stability. Summary of the invention
[0003] The present invention provides a method and device for automatic sample loading and unloading of a fully automatic AI microscopic analyzer, so as to solve the technical problems of low loading and unloading efficiency and poor loading and unloading stability in the prior art, and achieve the technical effect of improving loading and unloading efficiency and improving loading and unloading accuracy and stability.
[0004] In a first aspect, the present invention provides a method for automatically loading and unloading samples for a fully automatic AI microscopic analyzer, wherein the method comprises:
[0005] Based on the machine vision module, the image of the sample container is collected according to the two-step method and image recognition analysis is performed to determine the sample characteristics, wherein the sample characteristics include self-characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer; the transmission control characteristics of the loading and unloading mechanism are obtained, the control module is decoupled based on the control elements, and N sub-modules are determined, wherein the N sub-modules are relatively independent and collaborative; in combination with the N sub-modules, a sample-driven training gray decision model is performed, wherein the gray decision model includes an asynchronous loading and unloading decision unit and a process prediction unit; interactive loading and unloading task conditions, combined with the sample characteristics, execute the loading and unloading action sequence and path planning decision based on the loading and unloading decision unit, and determine the automatic loading and unloading strategy; the intelligent central control system responds to the automatic loading and unloading strategy, executes the automatic loading and unloading control of the sample container, and assists the machine vision module to synchronously record the execution data of the strategy node; combined with the process prediction unit, accompanying the strategy execution process, a gray prediction analysis is performed based on the execution data, and feedback adjustment of the loading and unloading of the sample container is performed.
[0006] In a second aspect, the present invention further provides a sample automatic loading and unloading device for a fully automatic AI microscopic analysis instrument, wherein the device comprises:
[0007] An image acquisition and recognition component is used to acquire images of sample containers based on a machine vision module and perform image recognition analysis according to a two-step method to determine sample characteristics, wherein the sample characteristics include autologous characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer.
[0008] A control characteristic acquisition component is used to acquire the transmission control characteristics of the loading and unloading mechanism, decouple the control module based on the control elements, and determine N sub-modules, wherein the N sub-modules are relatively independent and coordinated.
[0009] A decision model building component is used to combine the N sub-modules to perform sample-driven training of a grey decision model, wherein the grey decision model includes an asynchronous loading and unloading decision unit and a process prediction unit.
[0010] A task decision component is used to interact with loading and unloading task conditions, combine the sample features, execute loading and unloading action sequences and path planning decisions based on the loading and unloading decision unit, and determine automatic loading and unloading strategies.
[0011] A control and recording component, wherein the control and recording component is used for the intelligent central control system to respond to the automatic loading and unloading strategy, execute the automatic loading and unloading control of the sample container, and assist the machine vision module to synchronously record the execution data of the strategy node.
[0012] A feedback analysis and regulation component is used to combine with the process prediction unit, accompany the strategy execution process, perform grey prediction analysis based on the execution data, and perform feedback regulation of the sample container loading and unloading.
[0013] The present invention discloses a method and device for automatic sample loading and unloading of a fully automatic AI microscopic analyzer, comprising: based on a machine vision module, collecting an image of a sample container by a two-step method, and performing image recognition analysis to determine the characteristics of the sample, wherein the sample characteristics include self-characteristics and spatial distribution characteristics, wherein the spatial distribution characteristics are described with the microscopic analyzer as the center; obtaining the transmission control characteristics of the loading and unloading mechanism, decoupling the control module based on the control elements, and determining N relatively independent but collaborative submodules; combining these submodules, constructing a gray decision model for sample-driven training, wherein the gray decision model includes an asynchronously operated loading and unloading decision unit and a process prediction unit; interactive loading and unloading task conditions, combined with sample characteristics, executing loading and unloading action sequences and path planning based on the loading and unloading decision unit, and determining an automatic loading and unloading strategy; an intelligent central control system responds to the automatic loading and unloading strategy, executes automatic loading and unloading control of the sample container, and simultaneously assists the machine vision module to synchronously record the execution data of the strategy node; combined with the process prediction unit, during the strategy execution process, gray prediction analysis is performed based on the recorded execution data, and feedback adjustment and optimization are performed on the loading and unloading process of the sample container. The automatic sample loading and unloading method and device for a fully automatic AI microscopic analyzer disclosed in the present invention solve the technical problems of low loading and unloading efficiency and poor loading and unloading stability, and achieve the technical effect of improving loading and unloading efficiency and improving loading and unloading accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a schematic diagram of the process of the automatic sample loading and unloading method for the fully automatic AI microscopic analysis instrument of the present invention;
[0015] Figure 2 The figure is a schematic diagram of the structure of the automatic sample loading and unloading device used for the fully automatic AI microscopic analysis instrument of the present invention.
[0016] Explanation of the reference numerals: image acquisition and recognition component 11, control characteristic acquisition component 12, decision model building component 13, task decision component 14, control and recording component 15, feedback analysis and adjustment component 16. DETAILED DESCRIPTION
[0017] The technical solution provided in the embodiments of the present invention is to solve the technical problems of high latency, lack of real-time performance, incomplete monitoring, and insufficient predictability in the prior art. The overall idea adopted is as follows:
[0018] Firstly, based on the machine vision module, the image of the sample container is collected by a two-step method, and image recognition analysis is performed to determine the self-features and spatial distribution features of the sample, with the spatial distribution features centered on the microscopic analyzer. Secondly, the transmission control characteristics of the loading and unloading mechanism are obtained, and the control module is decoupled based on the control elements to determine N sub-modules, which are relatively independent and work together. Then, the N sub-modules are combined to perform a sample-driven training grey decision model, which includes an asynchronous loading and unloading decision unit and a process prediction unit. Next, the loading and unloading task conditions are interacted with, and combined with the sample characteristics, the loading and unloading action sequence and path planning decision based on the loading and unloading decision unit are executed to determine the automatic loading and unloading strategy. Then, the intelligent central control system responds to the automatic loading and unloading strategy, executes the automatic loading and unloading control of the sample container, and assists the machine vision module to synchronously record the execution data of the strategy node. Finally, combined with the process prediction unit, the strategy execution process is accompanied by a grey prediction analysis based on the execution data to complete the feedback adjustment of the loading and unloading of the sample container.
[0019] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention are shown in the drawings, rather than all of them. Example
[0020] Figure 1The present invention is a flow chart of a method for automatically loading and unloading samples for a fully automatic AI microscopic analyzer, wherein the method comprises:
[0021] Based on the machine vision module, the image of the sample container is collected according to the two-step method and image recognition analysis is performed to determine the sample characteristics, wherein the sample characteristics include autologous characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer.
[0022] Specifically, firstly, the machine vision module uses an image acquisition device (such as a camera, a webcam, etc.) to perform two-step image acquisition on the target sample container to obtain a multi-resolution image of the target sample container. The target sample container refers to a specific carrier for containing samples, and exemplarily includes special sample carriers such as a TIP head, a test paper, and a clip.
[0023] Furthermore, after acquiring the image of the sample container, the machine vision method is used to identify the characteristics of different types of sample containers and the sample distribution characteristics inside the container, and the output is the sample characteristics. Among them, the internal sample distribution characteristics correspond to the above-mentioned autologous characteristics, which refer to the physical properties of the sample itself, such as shape, color, texture, size, etc. For example, on the test paper, the autologous characteristics include the color change area of the test paper; in the TIP head, it may be the height of the liquid sucked from the sample; the sample container characteristics are the above-mentioned spatial distribution characteristics, that is, the position characteristics of the sample container, which are related to the position of the sample container and the microscopic analyzer.
[0024] Optionally, the analysis of spatial distribution characteristics is based on the relative position of the microscopic analyzer, which is the core equipment for testing samples. By aligning the relative position of the sample container and the microscopic analyzer, a unified reference coordinate is provided for testing the spatial arrangement of the sample in the container and whether it is in the correct position, which is convenient for subsequent testing operations.
[0025] The above method steps provide adaptability to conventional bottle sample containers and various sample carriers (such as TIP heads, test strips, clips, etc.) by respectively collecting autologous characteristics and spatial distribution characteristics, and have strong versatility.
[0026] In some embodiments, an image of a sample container is acquired according to a two-step method, comprising:
[0027] The machine vision module includes a scanning collector and an image recognizer; a first acquisition magnification and a second acquisition magnification are determined, and the sample container is scanned and acquired and feature recognition and extraction are performed in combination with the machine vision module to determine the sample features.
[0028] Specifically, the machine vision module includes two core components: a scanning collector and an image recognizer. The scanning collector is used to collect images of sample containers, and can obtain overall images and detailed images of sample containers at different magnifications. The scanning collector is a high-resolution camera or scanning device that can flexibly adjust the focal length and viewing angle at different acquisition magnifications. The image recognizer is used to process and analyze the collected images, identify and extract key features in the sample container. The image recognizer is built based on computer vision algorithms, exemplarily including edge detection, shape recognition, feature extraction and classification algorithms.
[0029] Specifically, the scanning collector and the image recognizer work together. First, the image data of the sample container is obtained by the scanning collector, and then the image recognizer analyzes the image to extract the autologous characteristics and spatial distribution characteristics of the sample. The scanning collector collects images of the target sample container based on the set first acquisition magnification and second acquisition magnification distribution. Through two image acquisition processes with different acquisition magnifications, the overall image and detail image of the sample container are obtained respectively, so as to more comprehensively and accurately identify the sample characteristics.
[0030] Specifically, the first acquisition magnification is used to obtain an overall image of the sample container to identify the overall appearance, position, shape and appearance characteristics of the sample container. The first acquisition magnification is low to ensure that the entire sample container can be covered. The second acquisition magnification is used to obtain a detailed image in the sample container, thereby facilitating the identification of the sample's autologous characteristics and spatial distribution characteristics. The second acquisition magnification is high in order to accurately capture detailed information inside the sample, such as the sample's texture, color change, distribution state, etc.
[0031] In some implementations, the machine vision module is combined to perform image scanning and feature recognition extraction on the sample container, including:
[0032] Based on the first acquisition magnification, the sample container is subjected to a low-magnification full-area rapid scan in combination with the scanning acquisition device to determine a blurred scan image; the image recognizer recognizes the blurred scan image to determine the spatial distribution characteristics of the sample container; based on the spatial distribution characteristics, the position of the sample container is located, and based on the second acquisition magnification, the sample container is subjected to a high-magnification automatic focusing acquisition to determine a sample scan image, and the image recognizer is used to recognize the autologous characteristics; the spatial distribution characteristics and the autologous characteristics are integrated to determine the sample characteristics.
[0033] Specifically, first, by determining the first acquisition magnification, the scanning collector obtains the full image of the sample container and outputs it as a fuzzy scanned image; then, the image recognizer performs feature recognition on the fuzzy scanned image to determine the basic self-features of the sample container such as shape, size, and material, thereby determining the category of the target sample container and providing the relative position of the target sample container in the microscopic analyzer. Low-magnification scanning can quickly capture the full picture of the sample container and is suitable for identifying the preliminary features of the sample and the position of the container in the working area. Exemplarily, the magnification can be 1-2 times, depending on the size and shape of the sample container.
[0034] Optionally, the image recognizer processes the blurred scanned image and determines its spatial distribution characteristics by identifying the edge, shape and position of the sample container. Exemplarily, first, the recognizer uses an edge detection algorithm (such as Sobel operator or Canny edge detection) to extract the outline of the sample container. Then, through a shape analysis algorithm (such as contour matching or Hough transform), the geometric features of the sample container are identified, such as whether it is circular, square or irregular in shape. Next, based on the acquired geometric features, the category of the sample container is determined, and the position coordinates of the sample container in the scanned image are specifically determined according to the category of the sample container, thereby determining its relative position in the working area of the microscopic analyzer.
[0035] Specifically, based on the spatial distribution characteristics identified in the previous step, the precise position of the sample container in the entire working area is located, and the focal length and angle of the scanning collector are adjusted to ensure that the high-magnification acquisition can focus on the center position of the sample container; then, switch to the second acquisition magnification and perform high-magnification automatic focus acquisition to capture the fine features inside the sample container, such as the color changes of the sample, texture details, liquid height, and the distribution status of the sample in the container.
[0036] For example, color space conversion (such as from RGB to HSV) and color segmentation techniques are used to identify color features in sample scanned images, such as detecting color changes on a test paper or the color of a sample liquid. Geometric features and surface details in sample scanned images are identified through shape recognition and texture analysis algorithms.
[0037] Furthermore, the identified spatial distribution features are integrated with the autologous features to form a complete sample feature description. The spatial distribution features provide the location and arrangement information of the sample in the container, while the autologous features provide the physical properties and detailed information of the sample.
[0038] The above method steps combine low-magnification full-area scanning and high-magnification automatic focusing acquisition, taking into account both the overall characteristics and fine characteristics of the sample container, and ensuring that the acquired image information is complete and accurate.
[0039] The transmission control characteristics of the loading and unloading mechanism are obtained, the control module is decoupled based on the control elements, and N sub-modules are determined, wherein the N sub-modules are relatively independent and coordinated.
[0040] Specifically, the loading and unloading mechanism includes a plurality of submodules, and the rotation action of sample unloading is realized through the mutual cooperation of the plurality of submodules. Each submodule corresponds to a relatively independent control task and independently processes a certain control element. Exemplarily, the N submodules of the loading and unloading mechanism include a clamping module, a supporting module, a limiting module, a power module, etc.
[0041] Specifically, the transmission control characteristics of the loading and unloading mechanism refer to the dynamic behavior and control elements of the mechanism when performing operations such as clamping, moving, loading and unloading. It involves motion control (such as position, speed, acceleration) and actuator control (such as clamping force, fixture position, etc.). Exemplarily, the transmission control characteristics can be obtained by accessing the loading and unloading mechanism design data, sensor data acquisition, system modeling and simulation analysis, including the motion characteristics and mechanical characteristics of the loading and unloading mechanism. Through the transmission control characteristics, the transmission characteristics of the loading and unloading mechanism can be fully understood.
[0042] Specifically, the control elements are the basic control parameters that affect the operation of the loading and unloading mechanism, such as position, movement speed, clamping force, fixture opening size, etc. There may be a coupling relationship between the control elements, so they need to be modularly decoupled to improve the flexibility and efficiency of the control system. By decoupling the control elements, complex control problems can be decomposed into multiple smaller sub-problems, so that each sub-problem can be processed and optimized independently, thereby reducing coupling interference in the control system and improving control accuracy. Exemplarily, the clamping operation can be decomposed into three modules: position control, fixture opening size control, and clamping force control. The size of the fixture opening may affect the size of the clamping force (different clamping force arms).
[0043] The above method steps ensure the accuracy and stability of the loading and unloading mechanism when performing complex operations through modular decoupling and independent collaborative control, reduce operational errors, and facilitate traceability analysis and feedback correction of operational errors.
[0044] In combination with the N sub-modules, a sample-driven training grey decision model is performed, wherein the grey decision model includes an asynchronous loading and unloading decision unit and a process prediction unit.
[0045] Specifically, the grey decision model is used to make control decisions for complex multivariables in loading and unloading mechanisms. The grey decision model includes asynchronous loading and unloading decision units and process prediction units to dynamically adjust the execution strategy of loading and unloading operations and perform predictive prediction and error correction based on real-time feedback.
[0046] Specifically, the asynchronous loading and unloading decision unit is used to determine the execution strategy of the entire loading and unloading cycle, relying on the grey model to process historical sample data and current environmental factors, make decisions, and dynamically adjust the execution strategy during the entire cycle. The process prediction unit is used to perform real-time detection during the execution of the decision unit, predict the next error risk based on the detection data, and make adjustments during the strategy execution process to ensure the accuracy and stability of the loading and unloading tasks.
[0047] In some embodiments, performing sample-driven training on a grey decision model includes:
[0048] Interact the coupled collaborative relationship of the N sub-modules; determine the unit architecture based on the N sub-modules; call the loading and unloading sample data, perform sample division and accumulation processing based on the unit architecture, and supervise the training of N modular branches; based on the coupled collaborative relationship, combine the N modular branches to determine the loading and unloading decision unit.
[0049] Specifically, a sample-driven training method is used to construct and train a grey decision model. First, the coupling and coordination relationship of N submodules is obtained, that is, it is determined which submodules have a mutually dependent or influential relationship. Exemplarily, by analyzing the data flow and control signals between the submodules, the interface and coordination mechanism between the submodules are established.
[0050] Exemplarily, to quantify the above coupling synergy relationship, if a certain action requires close cooperation between two sub-modules to be realized, then the coupling synergy relationship between the two sub-modules in the group is 1; if a certain action can be realized by a single sub-module, but adding another module can improve the action efficiency or accuracy, then the coupling synergy relationship between the two sub-modules in the group is a value between (0, 1).
[0051] Specifically, an overall architecture is designed based on the functions and interactions of the sub-modules. The positions and connections of each sub-module in the architecture are determined to ensure that the architecture can support the independent operation and collaborative work of each sub-module. Exemplarily, based on the functions and interactions of the sub-modules, the relationship edges of the sub-modules in the architecture are established, wherein different functional implementations correspond to different architecture call combinations, that is, N modular branches. In other words, the unit architecture is a collection of multiple combinations based on N sub-modules, which is used to realize loading and unloading decision-making and process prediction functions. Each action function of the loading and unloading mechanism corresponds to a modular branch, and the N modular branches form a complete loading and unloading control architecture.
[0052] In some implementations, sample partitioning and accumulation processing are performed based on the unit architecture, and supervised training of N modular branches includes:
[0053] Divide the loading and unloading sample data to determine N groups of sample data; traverse the N groups of sample data, perform sample accumulation processing on each group, and determine N groups of processed samples; perform sample-driven training based on the N groups of processed samples to obtain the N modular branches.
[0054] Specifically, in the control process of the loading and unloading mechanism, the unit architecture is composed of N submodules, each of which handles different control tasks, such as position control, clamping force control, fixture opening size, etc. In order to effectively train these modular branches, first, the sample data of the loading and unloading operations are divided into multiple groups, and processed and trained separately. By dividing and accumulating the loading and unloading sample data, an independent sample data group specifically applied to each modular branch can be established, and the control strategy of each modular branch can be obtained through sample-driven supervised training.
[0055] Optionally, the loading and unloading sample data are divided into N groups according to the different control tasks of the N submodules. For example, the sample data set corresponding to the position control module may include position sensor data, moving path information, etc.; the sample data set corresponding to the clamping force control module includes clamping force sensor data and related operation records.
[0056] Specifically, the divided N groups of sample data are processed one by one. The tasks of each modular branch are different, so the sample data they need are also different. By processing the sample data one by one, it can be ensured that the sample data of each modular branch is specifically for its task, reducing the cross-interference of data.
[0057] Optionally, in actual loading and unloading operations, there may be incomplete data collection, missing data, or insufficient sample size, which may result in the data samples of the training model not being able to fully cover all operation scenarios. Therefore, new sample data is generated through sample processing methods such as accumulation and subtraction, so that the data presents regularity, thereby meeting the sample data requirements for subsequent grey prediction model training.
[0058] Exemplarily, regular series are extracted from existing limited sample data through operations such as addition, subtraction, and sliding window processing to form new sample groups, thereby increasing the diversity and continuity of the data, thereby enhancing the training effect of the model when the amount of data is insufficient.
[0059] Furthermore, based on the processed sample data, each modular branch is trained through supervised learning, so that N modular branches can acquire the decision-making tasks of loading and unloading strategies in automatic handling of loading and unloading operations. For example, the clamping force control modular branch learns how to adjust the clamping force according to the sensor data through the corresponding processing sample group, the position control modular branch learns how to adjust the motion path according to the target position through the corresponding processing sample group, and the fixture opening control modular branch is trained based on the processed sample data on how to adjust the opening size of the fixture to adapt to sample containers of different sizes.
[0060] Through the above sample-driven supervised training, the N modular branches formed can automatically perform their corresponding loading and unloading tasks. This independence allows each modular branch to be flexibly adjusted during execution and optimized for specific tasks.
[0061] The interactive loading and unloading task conditions are combined with the sample features to execute the loading and unloading action sequence and path planning decision based on the loading and unloading decision unit to determine the automatic loading and unloading strategy.
[0062] Specifically, the loading and unloading task conditions refer to various parameters and requirements involved in performing loading and unloading operations. For example, they include the upper limit of the acceleration of the sample container, the upper limit of the speed, the upper limit of the loading and unloading time, the set of loading and unloading target locations, etc. The loading and unloading task conditions constrain the selection of the loading and unloading path, the arrangement of the operation steps, and the accuracy of the action.
[0063] Specifically, the loading and unloading task conditions and sample characteristics are the input data of the loading and unloading decision unit. The loading and unloading decision unit formulates the action sequence and path planning decisions in the loading and unloading process according to the loading and unloading task conditions and sample characteristics, and outputs the automatic loading and unloading strategy. Among them, the loading and unloading action sequence refers to the execution order of each step in the loading and unloading operation process. For example, the object is first clamped with a specific clamping force, then moved to the specified position, and finally the loading and unloading is completed. Path planning refers to the specific action trajectory and corresponding action parameters (such as speed, acceleration, etc.) between the above-mentioned loading and unloading action sequences to ensure that the object remains safe and stable during the handling process.
[0064] The above method steps organically combine loading and unloading task conditions, sample characteristics, action sequences and path planning to formulate automated loading and unloading strategies. Through the automated decision-making process, the technical effect of reducing operation time and improving loading and unloading accuracy is achieved. In addition, the dynamic adjustment capability based on the gray prediction model enables the system to continue to operate efficiently under complex or changing conditions.
[0065] In some embodiments, after determining the automatic loading and unloading strategy, the method further comprises:
[0066] The transmission control characteristics are traversed to determine sensitive control factors, wherein the sensitive control factors are marked with sensitivity coefficients; a loading and unloading safety distance is set; the automatic loading and unloading strategy is compensated based on the sensitive control factors, and loading and unloading avoidance constraints are performed based on the loading and unloading safety distance.
[0067] Specifically, sensitive control factors refer to control factors that affect the accuracy and stability of action execution, and thus affect sample quality, sample safety or sample analysis accuracy. Examples include clamping force, acceleration, tilt angle, etc. For example, too much clamping force may damage the object, while too little clamping force may cause the object to fall. Similarly, parameters such as position accuracy and speed may also become sensitive control factors.
[0068] Optionally, the sensitivity coefficient is used to quantify the degree of influence of each sensitive control factor on the operation result, which is determined by experimental data or model analysis. The larger the sensitivity coefficient, the more significant the influence of the factor on the loading and unloading operation, which requires special attention and fine control. In other words, the sensitivity coefficient determines the maximum tolerance range of the response sensitive indicator.
[0069] Specifically, the loading and unloading safety distance refers to the minimum distance between the loading and unloading mechanism and other equipment, workpieces or obstacles during the loading and unloading process to prevent collision, interference or operational errors. For example, the safety distance is set based on the characteristics of the loading and unloading task, the operating environment, the characteristics of the sample container, etc., to ensure that loading and unloading safety is maintained in a complex operating environment.
[0070] Optionally, large-sized objects require a greater safety distance to avoid collision with surrounding objects during transportation; mechanisms moving at higher speeds require a greater safety distance to ensure that they can stop in time and avoid collision in an emergency.
[0071] Specifically, after determining the sensitive control factors and their sensitivity coefficients, the initial automatic loading and unloading strategy is compensated and adjusted, including adjusting operating parameters such as position, speed, and clamping force to ensure that these sensitive factors can be better dealt with during the loading and unloading process. At the same time, when performing loading and unloading operations, the motion path, speed, or clamping action is adjusted based on the loading and unloading safety distance to avoid collision or interference and ensure that the operation is carried out within a safe range. Exemplary, it includes setting corresponding taboo spaces according to the loading and unloading safety distance and updating the path.
[0072] The intelligent central control system responds to the automatic loading and unloading strategy, executes the automatic loading and unloading control of the sample container, and assists the machine vision module to synchronously record the execution data of the strategy node.
[0073] Specifically, the intelligent central control system receives the generated automatic loading and unloading strategy and converts it into the corresponding machine execution code (such as g-code, etc.) to perform automatic loading and unloading control of the container. At the same time, the machine vision module is activated to synchronously collect the motion trajectory and motion posture data of the sample container, and obtain the execution data of the strategy node to help identify deviations and problems in the operation, and provide a basis for subsequent predictions and adjustments. Among them, the strategy nodes include specific operation moments such as grasping, handling, loading and unloading.
[0074] In combination with the process prediction unit, along with the strategy execution process, grey prediction analysis is performed based on the execution data to perform feedback regulation of the sample container loading and unloading.
[0075] Specifically, the execution data is used to characterize loading and unloading quality information such as whether the sample container is accurately grasped, the stability during the transportation process, and whether the final loading and unloading reaches the predetermined position.
[0076] Specifically, the process prediction unit analyzes the strategy execution data in real time, and performs predictive analysis and risk assessment on the loading and unloading process based on the grey prediction model. First, the process prediction unit predicts the automatic loading and unloading strategy to obtain the ideal execution status data, and then obtains the real-time execution status data along with the strategy execution process, and performs path fitting and similarity analysis with the above-mentioned ideal execution status data to determine the grey prediction analysis result, which reflects the difference between the actual strategy execution and the ideal situation, and determines the error in the sample container loading and unloading execution process.
[0077] Specifically, when it is detected that certain parameters are beyond the normal range, the prediction unit will make adjustment suggestions based on the current grey prediction analysis results. For example, during loading and unloading, if the clamping force changes beyond the predicted range, the system can determine the possibility of clamping error and make timely adjustments.
[0078] Optionally, the grey prediction model identifies the changing trend and potential deviation of the data based on the preprocessed data, and makes forward-looking feedback adjustments. For example, if the position data continues to deviate, indicating that the loading and unloading error increases, the prediction unit will issue a warning and make corrections accordingly, adjusting the motion path of the fixture to avoid error amplification or task failure.
[0079] The above method steps achieve the technical effect of continuously optimizing the operation strategy during the loading and unloading process and ensuring the precise execution of loading and unloading tasks through the response of the intelligent central control system, the data recording of the machine vision module and the predictive analysis of the grey prediction model.
[0080] Furthermore, the method further comprises:
[0081] After the sample loading is completed, the loading record data is generated; the sample unloading position is determined, and if it is a recovery unloading, the loading record data is traversed for reverse conversion, and the unloading control management of the sample container is performed.
[0082] Specifically, if the sample container needs to be restored and unloaded after the sample loading is completed, even if the sample container returns to the initial position, the sample container will be returned to the original loading position in reverse order according to the order and method of loading, or the original operation path will be simulated. Among them, the loading record data includes the loading position of the sample, the size, shape, clamping force and other key operation parameters of the sample container. At the same time, the operation sequence and time point during the loading process are also recorded. By performing reverse operation control on the loading record data, that is, performing the unloading task in the opposite order and operation, the sample container can be restored and unloaded under the premise of ensuring the accuracy and safety of the unloading process.
[0083] In summary, the automatic sample loading and unloading method for the fully automatic AI microscopic analyzer provided by the present invention has the following technical effects:
[0084] Based on the machine vision module, the image of the sample container is collected through a two-step method, and image recognition analysis is performed to determine the characteristics of the sample. The sample characteristics include self-characteristics and spatial distribution characteristics, among which the spatial distribution characteristics are described with the microscopic analyzer as the center; the transmission control characteristics of the loading and unloading mechanism are obtained, the control module is decoupled based on the control elements, and N relatively independent but collaborative sub-modules are determined; these sub-modules are combined to construct a gray decision model for sample-driven training, wherein the gray decision model includes an asynchronously operated loading and unloading decision unit and a process prediction unit; interactive loading and unloading task conditions, combined with sample characteristics, based on the loading and unloading decision unit, the loading and unloading action sequence and path planning are executed to determine the automatic loading and unloading strategy; the intelligent central control system responds to the automatic loading and unloading strategy, executes the automatic loading and unloading control of the sample container, and assists the machine vision module to synchronously record the execution data of the strategy node; combined with the process prediction unit, during the strategy execution process, gray prediction analysis is performed based on the recorded execution data, and the loading and unloading process of the sample container is feedback-adjusted and optimized. Thus, the technical effect of improving loading and unloading efficiency and improving loading and unloading accuracy and stability is achieved. Example
[0085] Figure 2 Schematic diagram of the structure of the automatic sample loading and unloading device for the fully automatic AI microscopic analysis instrument of the present invention. Figure 1 The flow chart of the automatic sample loading and unloading method for the fully automatic AI microscopic analysis instrument of the present invention can be shown as follows: Figure 2 The structure shown is implemented.
[0086] Based on the same concept as the automatic sample loading and unloading method for the fully automatic AI microscopic analysis instrument in the above embodiment, the present invention also provides an automatic sample loading and unloading device for the fully automatic AI microscopic analysis instrument, comprising:
[0087] The image acquisition and recognition component 11 is used to acquire the image of the sample container based on the machine vision module and perform image recognition analysis according to the two-step method to determine the sample characteristics, wherein the sample characteristics include autologous characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer.
[0088] The control characteristic acquisition component 12 is used to acquire the transmission control characteristics of the loading and unloading mechanism, decouple the control module based on the control elements, and determine N submodules, wherein the N submodules are relatively independent and coordinated.
[0089] The decision model building component 13 is used to combine the N sub-modules to perform sample-driven training of a grey decision model, wherein the grey decision model includes an asynchronous loading and unloading decision unit and a process prediction unit.
[0090] The task decision component 14 is used to interact with the loading and unloading task conditions, combine the sample features, execute the loading and unloading action sequence and path planning decision based on the loading and unloading decision unit, and determine the automatic loading and unloading strategy.
[0091] The control and recording component 15 is used for the intelligent central control system to respond to the automatic loading and unloading strategy, execute the automatic loading and unloading control of the sample container, and assist the machine vision module to synchronously record the execution data of the strategy node.
[0092] The feedback analysis and adjustment component 16 is used to combine with the process prediction unit, accompany the strategy execution process, perform grey prediction analysis based on the execution data, and perform feedback adjustment of the sample container loading and unloading.
[0093] The image acquisition and recognition component 11 includes:
[0094] A machine vision module, wherein the machine vision module comprises a scanning collector and an image recognizer.
[0095] The image scanning acquisition and feature recognition and extraction module is used to determine the first acquisition magnification and the second acquisition magnification, and in combination with the machine vision module, performs image scanning acquisition and feature recognition and extraction on the sample container to determine the sample features.
[0096] In some implementations, the image scanning acquisition and feature recognition extraction module in the image acquisition and recognition component 11 includes:
[0097] The full-area scanning module is used to perform a low-magnification full-area rapid scan on the sample container based on the first acquisition magnification in combination with the scanning collector to determine a blurred scan image.
[0098] The image recognition and feature extraction module is used to recognize the blurred scan image with the image recognizer and determine the spatial distribution characteristics of the sample container.
[0099] The positioning and high-magnification acquisition module is used to locate the position of the sample container based on the spatial distribution characteristics, perform high-magnification automatic focusing acquisition on the sample container based on the second acquisition magnification, determine the sample scanning image, and identify the autologous characteristics in combination with the image recognizer.
[0100] The self-feature recognition and feature integration module is used to integrate the spatial distribution features and the self-features to determine the sample features.
[0101] In some embodiments, the decision model building component 13 includes:
[0102] The collaborative relationship interaction module is used to interact the coupling collaborative relationship of the N sub-modules.
[0103] The unit architecture determination module is used to determine the unit architecture based on the N sub-modules.
[0104] The data processing and branch training module is used to call and load sample data, perform sample division and cumulative processing based on the unit architecture, and supervise the training of N modular branches.
[0105] The loading and unloading decision unit joint determination module is used to determine the loading and unloading decision unit based on the coupling collaborative relationship and the N modular branches.
[0106] In some implementations, the data processing and branch training module in the decision model building component 13 includes:
[0107] The sample data division module is used to divide the loading and unloading sample data to determine N groups of sample data.
[0108] The sample accumulation processing module is used to traverse the N groups of sample data, perform sample accumulation processing on each group, and determine N groups of processed samples.
[0109] The sample-driven training and modular branch acquisition module is used to perform sample-driven training based on the N groups of processed samples to acquire the N modular branches.
[0110] In some embodiments, the task decision component 14 includes:
[0111] The sensitive control factor identification module is used to traverse the transmission control characteristics and determine sensitive control factors, wherein the sensitive control factors are identified with sensitivity coefficients.
[0112] The loading and unloading safety distance setting module is used to set the loading and unloading safety distance.
[0113] The automatic loading and unloading strategy compensation and avoidance constraint module is used to compensate the automatic loading and unloading strategy based on the sensitive control factors and to perform loading and unloading avoidance constraints based on the loading and unloading safety distance.
[0114] Furthermore, the device also includes:
[0115] The record generation module is used to generate loading record data after completing sample loading.
[0116] The feedback control module is used to determine the sample unloading position. If it is a recovery unloading, the loading record data is traversed for reverse conversion, and the unloading control management of the sample container is performed.
[0117] It should be understood that the embodiments mentioned in this specification focus on their differences from other embodiments. The specific embodiments in the aforementioned embodiment 1 are also applicable to the automatic sample loading and unloading device for the fully automatic AI microscopy analyzer described in embodiment 2. For the sake of brevity of the specification, no further elaboration is given here.
[0118] It should be understood that the embodiments disclosed in the present invention and the above description can enable those skilled in the art to use the present invention to implement the present invention. At the same time, the present invention is not limited to the above-mentioned embodiments. It should be understood that those skilled in the art can still modify the technical solutions recorded in the above-mentioned embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. A method for automatically loading and unloading samples for a fully automatic AI microscopic analyzer, characterized in that: The method comprises: Based on the machine vision module, the image of the sample container is collected according to the two-step method and image recognition analysis is performed to determine the sample characteristics, wherein the sample characteristics include autologous characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer; Acquire the transmission control characteristics of the loading and unloading mechanism, decouple the control module based on the control elements, and determine N submodules, wherein the N submodules are relatively independent and coordinated; In combination with the N submodules, a sample-driven training grey decision model is performed, wherein the grey decision model comprises an asynchronous loading and unloading decision unit and a process prediction unit; Interactive loading and unloading task conditions, combined with the sample features, execute loading and unloading action sequences and path planning decisions based on the loading and unloading decision unit, and determine automatic loading and unloading strategies; The intelligent central control system responds to the automatic loading and unloading strategy, executes the automatic loading and unloading control of the sample container, and assists the machine vision module to synchronously record the execution data of the strategy node; In combination with the process prediction unit, along with the strategy execution process, grey prediction analysis is performed based on the execution data to perform feedback regulation of the sample container loading and unloading; Wherein, the collecting of the image of the sample container according to the two-step method comprises: The machine vision module includes a scanning collector and an image recognizer; Determine a first acquisition magnification and a second acquisition magnification, and perform image scanning and acquisition and feature recognition and extraction on the sample container in combination with the machine vision module to determine the sample features; In which, combining the machine vision module, performing image scanning and acquisition and feature recognition and extraction on the sample container includes: Based on the first acquisition magnification, in combination with the scanning collector, a low-magnification full-area rapid scan is performed on the sample container to determine a blurred scan image; The image recognizer recognizes the blurred scan image and determines the spatial distribution characteristics of the sample container; Based on the spatial distribution characteristics, locate the position of the sample container, based on the second acquisition magnification, perform high-magnification automatic focusing acquisition on the sample container, determine the sample scanning image, and identify the autologous characteristics in combination with the image recognizer; The spatial distribution characteristics and the autologous characteristics are integrated to determine the sample characteristics.
2. The automatic sample loading and unloading method for a fully automatic AI microscopic analyzer according to claim 1, characterized in that: Conduct sample-driven training of grey decision-making model, including: Interacting the coupling and coordination relationship of the N submodules; Based on the N sub-modules, determining a unit architecture; Calling loading and unloading sample data, performing sample division and accumulation processing based on the unit architecture, and supervising the training of N modular branches; Based on the coupling cooperative relationship, the N modular branches are combined to determine the loading and unloading decision unit.
3. The automatic sample loading and unloading method for a fully automatic AI microscopic analyzer according to claim 2, characterized in that: Based on the unit architecture, sample division and accumulation processing are performed, and N modular branches are supervised for training, including: Dividing the loading and unloading sample data to determine N groups of sample data; Traversing the N groups of sample data, performing sample accumulation processing on each group, and determining N groups of processed samples; Based on the N groups of processed samples, sample-driven training is performed to obtain the N modular branches.
4. The automatic sample loading and unloading method for a fully automatic AI microscopic analyzer according to claim 1, characterized in that: After the automatic loading and unloading strategy is determined, the following steps are included: Traversing the transmission control characteristics to determine sensitive control elements, wherein the sensitive control elements are marked with sensitivity coefficients; Set safe distance for loading and unloading; The automatic loading and unloading strategy is compensated based on the sensitive control factors, and loading and unloading avoidance constraints are performed based on the loading and unloading safety distance.
5. The method for automatically loading and unloading samples for a fully automatic AI microscopic analyzer according to claim 1, characterized in that the method further comprises: After completing sample loading, generate loading record data; The sample unloading position is determined. If it is a recovery unloading, the loading record data is traversed for reverse conversion, and the unloading control management of the sample container is performed.
6. A sample automatic loading and unloading device for a fully automatic AI microscopic analyzer, characterized in that: The device is used to perform the automatic sample loading and unloading method for a fully automatic AI microscopic analysis instrument according to any one of claims 1 to 5, and the device comprises: An image acquisition and recognition component, which is used to acquire an image of a sample container based on a machine vision module and perform image recognition analysis according to a two-step method to determine sample characteristics, wherein the sample characteristics include autologous characteristics and spatial distribution characteristics, and the spatial distribution characteristics are centered on the microscopic analyzer; A control characteristic acquisition component, the control characteristic acquisition component is used to acquire the transmission control characteristics of the loading and unloading mechanism, decouple the control module based on the control elements, and determine N submodules, wherein the N submodules are relatively independent and coordinated; A decision model building component, wherein the decision model building component is used to combine the N sub-modules to perform sample-driven training of a grey decision model, wherein the grey decision model includes an asynchronous loading and unloading decision unit and a process prediction unit; A task decision component, the task decision component is used to interact with loading and unloading task conditions, combine the sample features, execute loading and unloading action sequences and path planning decisions based on the loading and unloading decision unit, and determine automatic loading and unloading strategies; A control and recording component, wherein the control and recording component is used for the intelligent central control system to respond to the automatic loading and unloading strategy, execute the automatic loading and unloading control of the sample container, and assist the machine vision module to synchronously record the execution data of the strategy node; A feedback analysis and regulation component is used to combine with the process prediction unit, accompany the strategy execution process, perform grey prediction analysis based on the execution data, and perform feedback regulation of the sample container loading and unloading.
Citation Information
Patent Citations
Object loading implementation method, device, system and computer equipment
CN111652558A
Sample centrifugal management system and method for intelligent laboratory
CN112034191A
