Instrument packaging table monitoring robot based on AI picture recognition function and monitoring method

Abstract

The invention relates to the technical field of intelligent monitoring, and discloses an instrument packaging table monitoring robot based on an AI picture recognition function and a monitoring method. The monitoring robot comprises a data acquisition unit, a preprocessing unit, a judgment early warning unit, an operation detection unit, a multi-point scheduling unit, an evaluation analysis unit and the like. Partition environment data of an instrument packaging table are collected through multiple sensors, after preprocessing, the judgment early warning unit compares the data with a standard value, and the operation detection unit recognizes operation normalization and speed. The multi-point scheduling unit reasonably deploys the detection equipment, and the evaluation analysis unit calculates a difference parameter to generate an operation stability evaluation value and executes a corresponding strategy according to the operation stability evaluation value. Accurate monitoring, intelligent early warning and resource optimization of operation of the instrument packaging table are achieved, and the medical instrument packaging quality, the production efficiency and the management level are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and particularly to a monitoring robot and a monitoring method for an instrument packing table based on the AI image recognition function. Background Art

[0002] In the production and supply links of medical devices, the instrument packing operation is crucial, and its operation quality directly affects the safety and effectiveness of medical devices. There are many drawbacks in the traditional monitoring methods for instrument packing tables.

[0003] From the perspective of human monitoring, manual monitoring relies on the subjective judgment and experience of operators, and is extremely vulnerable to factors such as fatigue and emotions. Long-term high-intensity work will cause the attention of operators to decline, making it difficult to ensure that every packing operation step is carefully observed, resulting in some non-standard operations not being discovered in time. For example, during the packing process, problems such as inaccurate placement of instruments and uneven packing force may occur, and manual monitoring may miss inspections, thus affecting the packaging quality of medical devices and posing potential safety hazards.

[0004] In terms of automated monitoring, early automated devices could only monitor simple parameters, such as the statistics of the number of packs, and could not analyze in depth the details of the operation process. With the development of technology, although some automated monitoring systems have been improved, their functions are still limited. They may only have a single monitoring function, such as only being able to monitor the position of instruments, and cannot simultaneously monitor the operation force and the standardization of operation actions, making it difficult to comprehensively evaluate the quality of packing operations.

[0005] Traditional monitoring methods lack in-depth analysis and utilization of data. Even if some data is collected, it is only simply stored, without effective integration and analysis of the data, and potential problems and rules cannot be mined from the data. For example, by comparing the operation data of different time periods and different operators, operation differences and optimization directions cannot be found, and it is difficult to achieve continuous improvement of the entire packing process.

[0006] There is often a lack of effective coordination and linkage between different monitoring devices. In actual production, multiple monitoring devices may act independently and cannot achieve data sharing and collaborative work. This leads to the inability to quickly locate the root cause of problems when problems are found, and it is also difficult to take targeted measures to solve them, seriously affecting production efficiency and product quality.

[0007] With the continuous improvement of the requirements for product quality and safety in the medical device industry, the traditional monitoring methods for instrument packing tables can no longer meet the needs. Therefore, it is urgent to develop a monitoring system for instrument packing tables that can achieve all-round, accurate, and intelligent monitoring, which not only helps to improve the packaging quality of medical devices, ensure the safety of patients, but also can improve the production efficiency and management level of enterprises. Summary of the Invention

[0008] The purpose of the present invention is to provide a monitoring robot and a monitoring method for an instrument packing table based on the AI image recognition function, so as to solve the problems proposed in the above background technology.

[0009] To achieve the above purpose, the present invention provides the following technical solutions: A monitoring robot for an instrument packing table based on the AI image recognition function, including:

[0010] A data acquisition unit that detects the instrument packing table in partitions, uses detection equipment to obtain environmental data of each partition, and the detection equipment is configured in each partition, including a visual sensor group, a position sensor, and an operating force sensor;

[0011] A preprocessing unit that preprocesses the environmental data obtained from each partition;

[0012] A determination and warning unit that compares each data in the environmental data with the preset standard range value of the corresponding partition. If the corresponding data is within the corresponding standard range value, no response action is taken. Otherwise, a warning is issued;

[0013] An operation detection unit that extracts the image data collected from the visual sensor group, processes and calculates the image data to obtain the standardization and operation speed of the instrument packing operation in the corresponding partition. The process of processing and calculating the image data includes the recognition of the instrument packing operation steps. When the steps are recognized, the rule engine in the growth detection unit is triggered to mark the images that do not conform to the standard operation steps;

[0014] A multi-point scheduling unit that, under the condition that the standardization of the instrument packing operation in a certain partition meets the requirements, drives the detection equipment in that partition to the partition with images marked with non-standard operation steps to obtain the environmental data in the partition with images marked with non-standard operation steps again;

[0015] An evaluation and analysis unit that calculates the difference parameter of the environmental data in the same partition obtained by two detection devices, builds a data analysis model based on the difference parameter to generate the operation stability evaluation value W czzi of the corresponding partition, and compares the operation stability evaluation value W czzi with the preset evaluation threshold, and executes corresponding strategies according to the comparison results.

[0016] Preferably, the vision sensor group consists of a high-definition camera and a depth camera, and the vision sensor group is of an integral structure and is arranged on an electric control track above the packing table. The vision sensor group is used to collect the position images and operation action images of the instruments in the corresponding partition in the environmental data. The i in the partition environmental data obtained by the detection device represents the partition number, and the partition numbers are marked in a clockwise order, i = 1, 2,... n, where n is a positive integer.

[0017] Preferably, the position sensor is arranged on the electric control track configured around the packing table to monitor the position of the instruments in the corresponding partition in real time from the horizontal and vertical directions.

[0018] Preferably, the induction end of the operation force sensor is always in contact with the packing tool, and is connected to the electric control track configured on the periphery through an L-shaped metal rod. The operation force sensor is used to collect the operation force magnitude during the packing operation in the corresponding partition in the environmental data.

[0019] Preferably, the steps for preprocessing the environmental data obtained for each partition are as follows:

[0020] S101. Clean the environmental data to remove fuzzy data, error data, and redundant data;

[0021] S102. Summarize the environmental data in the same partition to form an environmental data set for the corresponding partition.

[0022] Preferably, the steps for processing and calculating the image data are as follows:

[0023] S201. Feature extraction: Extract the features representing the instrument packing operation steps from the image data;

[0024] S202. Step matching: Use the template matching algorithm to match the extracted features with the preset standard operation step template;

[0025] S203. Normative judgment: Judge whether the operation conforms to the norm according to the matching result, compare the execution situation of the operation steps in the image data at the same time interval to obtain the normative difference, and trigger the rule engine in the growth detection unit when performing the normative judgment;

[0026] S204. Operation speed estimation: Calculate the operation speed of the instrument packing by dividing the number of operation steps completed between adjacent time points by the time interval.

[0027] Preferably, calculate the difference parameters of the environmental data in the same partition obtained by two detection devices, where the difference parameters include the position difference C wz , the operation force difference C cl , and the image similarity difference C xt, where the position difference C wz = |the position of the instrument W obtained by the detection device in this area zi - the position of the instrument W obtained by the scheduled detection device z |. For other differences in the difference parameters, the same method as the position difference C wz is used for calculation and acquisition.

[0028] Preferably, the steps for generating the operation stability evaluation value W czzi for the corresponding partition are as follows:

[0029] S301. Normalize the position difference C wz , operation force difference C cl , and image similarity difference C xt in the difference parameters;

[0030] S302. Build a data analysis model to generate the operation stability evaluation value W czzi for the corresponding partition. The formula is as follows:

[0031] W czzi = b1C wz + b2C cl + b3C xt + H

[0032] In the formula, b1, b2, and b3 are the proportionality coefficients of the position difference C wz , operation force difference C cl , and image similarity difference C xt respectively, and b1 > b2 > b3 > 0, and H is a constant correction coefficient.

[0033] Preferably, the result of comparing the operation stability evaluation value W czzi with the preset evaluation threshold is as follows:

[0034] If the operation stability evaluation value W czzi does not exceed the evaluation threshold, no response action is taken. If the operation stability evaluation value W czzi exceeds the evaluation threshold, alarm processing is performed and an adjustment strategy is executed. Among them, the alarm processing is to perform sound and light alarm through setting an alarm, which is used to form a prompt for the execution of the adjustment strategy. The adjustment strategy is to check and calibrate the packing tool for the corresponding partition and remind the operator of the operation specifications.

[0035] Preferably, the present invention further includes a monitoring method applied to the above robot, including the following steps:

[0036] Data acquisition: Use the data acquisition unit to collect environmental data for each partition of the instrument packing table;

[0037] Data preprocessing: The collected environmental data is cleaned and summarized by a preprocessing unit;

[0038] Judgment and early warning: The judgment and early warning unit compares the environmental data with the preset standard range value, and issues an early warning if it exceeds the range;

[0039] Operation detection: The operation detection unit processes and calculates the image data collected by the visual sensor group, identifies the operation standardization, and marks the images that do not meet the specifications;

[0040] Multi-point scheduling: When the operation standardization in the partition meets the requirements, the detection device is scheduled to the partition with marked images;

[0041] Evaluation and analysis: Calculate the difference parameters, build a data analysis model to generate an operation stability evaluation value, and execute corresponding strategies after comparing with the evaluation threshold.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] Through the visual sensor group, position sensor and operation force sensor in the data acquisition unit of the present invention, environmental data of each partition of the instrument packing table is obtained from multiple dimensions. The high-definition camera and depth camera of the visual sensor group can clearly capture the instrument position image and operation action image, providing rich visual information for judging the operation standardization; the position sensor monitors the instrument position in real time from horizontal and vertical directions to ensure that the instrument operates at the correct position; the operation force sensor collects the magnitude of the packing operation force in real time to avoid affecting the packing quality due to improper operation force. These sensors work together to achieve comprehensive and accurate monitoring of the operation process. Compared with traditional monitoring methods, problems in the operation can be found more timely and accurately.

[0044] The judgment and early warning unit compares the environmental data with the preset standard range value, and immediately issues an early warning once the data exceeds the range. The operation detection unit uses AI image recognition technology to process and calculate the image data, identify the operation standardization, and mark the images that do not meet the specifications. When the operation stability evaluation value exceeds the preset threshold, the evaluation and analysis unit will trigger an alarm and execute adjustment strategies, including checking and calibrating the packing tools and reminding the operators of the operation specifications. This intelligent abnormal handling mechanism can intervene at the first time when problems occur, effectively reducing the probability of unqualified products and ensuring the packaging quality of medical devices.

[0045] When the operation standardization in the partition meets the requirements, the multi-point scheduling unit schedules the detection device to the partition with non-standard operation marks. This function realizes the reasonable allocation of the detection device, avoids over-monitoring of the normal operation area, concentrates resources on the problem area, and improves the monitoring efficiency. At the same time, through key monitoring of the abnormal area, the cause of the problem can be analyzed more deeply, providing a basis for optimizing the operation process.

[0046] The evaluation and analysis unit builds a data analysis model by calculating difference parameters and generates an operation stability evaluation value. These data provide enterprises with quantitative operation evaluation indicators. Enterprises can conduct in-depth analysis based on these data to understand the operation differences of different operators and different time periods, identify the weak links in operations, and formulate targeted training plans and improvement measures. Through the accumulation and analysis of historical data, the preset standard range values and evaluation thresholds can also be continuously optimized, making the monitoring system more in line with actual production needs and achieving continuous improvement of the packing operation process.

[0047] The monitoring robot and monitoring method of the present invention achieve automatic and intelligent monitoring of the operation of the instrument packing table, reduce the workload and error of manual monitoring, and improve production efficiency. At the same time, various data reports and analysis results generated by the system provide strong support for enterprise management, helping managers comprehensively understand the production situation, make more scientific decisions, and thus improve the overall management level of the enterprise. Brief Description of the Drawings

[0048] Figure 1 is the working principle diagram of the instrument packing table monitoring robot based on the AI image recognition function of the present invention;

[0049] Figure 2 is the flow chart of environmental data preprocessing;

[0050] Figure 3 is the working principle diagram of image data processing and calculation;

[0051] Figure 4 is the working principle diagram of difference parameter calculation. Detailed Embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Please refer to Figures 1-4 , the present invention provides a technical solution: an instrument packing table monitoring robot based on the AI image recognition function, the core of which is to achieve comprehensive, accurate and intelligent monitoring of the operation of the instrument packing table. Its overall implementation relies on the coordinated operation of the following multiple functional units:

[0054] Data acquisition unit: This unit undertakes the important task of obtaining environmental data of each partition on the instrument packing table. By deploying detection devices in each partition, including a visual sensor group, a position sensor, and an operating force sensor, data is collected from different dimensions. The visual sensor group is used to capture the position images and operating action images of the instruments; the position sensor is responsible for monitoring the position information of the instruments in the horizontal and vertical directions; the operating force sensor real-time collects the magnitude of the operating force during the packing operation, so as to obtain the environmental data of each partition comprehensively.

[0055] Preprocessing unit: Its main responsibility is to preprocess the environmental data obtained by the data acquisition unit. By removing the fuzzy data, error data, and redundant data in it, the data quality is improved. At the same time, the data of the same partition is summarized to form the corresponding environmental data set, providing a reliable data basis for subsequent analysis and processing.

[0056] Judgment and early warning unit: This unit compares each item of data in the environmental data with the preset standard range values of the corresponding partition. If the data is within the standard range, it indicates that the current operation is normal and no additional operation is required; if the data exceeds the standard range, an early warning signal will be sent in time to remind relevant personnel to pay attention to the abnormal operation situation.

[0057] Operation detection unit: This unit starts from the image data collected by the visual sensor group, and through a series of processing and calculations, obtains the standardization and operation speed of the instrument packing operation. In this process, it is necessary to identify the instrument packing operation steps. Once an unqualified operation step is found, the rule engine in the growth detection unit is triggered to mark the corresponding image.

[0058] Multi-point scheduling unit: When the standardization of the instrument packing operation in a certain partition meets the requirements, this unit will drive the detection devices in this partition to the partition with the image marks of unqualified operation steps. In this way, the environmental data of this partition can be obtained again to further deeply analyze and evaluate the situation of the abnormal operation area.

[0059] Evaluation and analysis unit: This unit calculates the difference parameters of the environmental data in the same partition obtained by the two detection devices, builds a data analysis model based on these difference parameters, and then generates the operation stability evaluation value W of the corresponding partition czzi After that, the operation stability evaluation value W czzi is compared with the preset evaluation threshold, and corresponding strategies are executed according to the comparison results.

[0060] The present invention will be further described below in conjunction with Embodiments 1 to 6:

[0061] Embodiment 1:

[0062] This embodiment focuses on the content of sensor configuration and partition numbering, which aims to provide precise hardware support and orderly partition management for data collection, ensuring that the monitoring robot can accurately obtain the environmental data of each partition.

[0063] The vision sensor group, as a key component for data collection, consists of a high-definition camera and a depth camera, and is of an integral structure. It is installed on the electric control track above the packing table. This layout enables it to overlook each partition from above, comprehensively collecting the position images and operation action images of the instruments within the corresponding partition. With its high-resolution characteristics, the high-definition camera can clearly capture the details of the instruments, such as the shape and surface texture of the instruments. These details are crucial for determining whether the instruments are in the correct position and whether the operation actions are standardized. The depth camera, by obtaining the depth information of the images, provides the system with three-dimensional space perception ability, enabling the monitoring robot to more accurately judge the positional relationship of the instruments in space, such as the distance between the instrument and the edge of the packing table, and the relative positions between the instruments.

[0064] When partitioning the instrument packing table, an orderly numbering rule is adopted. The i in the partition environmental data obtained by the detection device represents the partition number, which is marked as i = 1, 2,... n in a clockwise order, where n is a positive integer. This numbering method facilitates the system to manage the data of each partition. During the data collection process, through the partition number, the specific partition can be quickly located, and the collected data can be accurately corresponded to the corresponding partition. For example, when the vision sensor group collects image data, the system can quickly determine which partition the image data comes from according to the partition number, facilitating subsequent data processing and analysis.

[0065] The position sensors are installed on the electric control tracks configured around the packing table, and they monitor the positions of the instruments in the corresponding partition in real time from both horizontal and vertical directions. Horizontally, the position sensors can accurately measure the coordinate positions of the instruments in the plane of the packing table. Once the instruments have a displacement deviation in the horizontal direction, such as deviating from the preset operation area, the position sensors will immediately capture this change and transmit the relevant data to the subsequent processing unit. Vertically, the position sensors can monitor the height changes of the instruments. For example, whether the height at which the instruments are lifted or lowered during the packing process meets the requirements of the standard operation. By monitoring the vertical position data, packing mistakes caused by improper instrument height can be effectively avoided.

[0066] The matching induction end of the operating force sensor is always in contact with the packing tool and is connected to the electronically controlled track configured peripherally through an L-shaped metal rod. During the packing operation, the operating force sensor can collect the magnitude of the operating force in real time during the packing operation. Different packing tasks and operating links require different operating forces. For example, a larger operating force is required when tightening the packing belt, while a smaller operating force is required when placing fragile items. The operating force sensor transmits the collected operating force data to the system in real time. When the operating force exceeds the preset standard range, the judgment and warning unit will issue a warning signal in time to prompt the operator to adjust the operating force to ensure the quality and safety of the packing operation.

[0067] Embodiment 2:

[0068] When preprocessing the obtained environmental data for each partition, the following steps are followed:

[0069] S101. Clean the environmental data: In the actual data collection process, due to the influence of various factors, the collected environmental data may have quality problems. Blurred data may be caused by poor light conditions, contaminated sensor lenses, etc., resulting in blurred image data. These data cannot accurately reflect the position and operation of the instrument and will interfere with subsequent analysis and judgment, so they need to be removed. Error data may result from sensor failures, data transmission errors, etc. For example, the sensor misreports the position of the instrument or the operating force value. If these error data are not processed, they will cause the system to make incorrect decisions, so they must be identified and removed through specific data cleaning algorithms. Redundant data refers to data that is repeated or has no actual contribution to the analysis result, such as the same position or the same operating force data collected multiple times. Excessive redundant data will occupy the storage space of the system and reduce the data processing efficiency, and also need to be cleaned up.

[0070] S102. Summarize the environmental data in the same partition: After the data is cleaned, it is classified and summarized according to the partition number. The environmental data in the same partition is integrated together to form an environmental data set corresponding to the partition. This is conducive to the centralized management and analysis of the data in each partition. For example, statistical analysis can be performed on the environmental data set of each partition to calculate statistical indicators such as the average value of the instrument position and the change range of the operating force, so as to understand the overall situation and characteristics of the operation in that partition. At the same time, the summarized data is also convenient for comparative analysis with the data of other partitions, discovering the differences and potential problems between different partitions, and providing data support for optimizing the packing operation process.

[0071] Embodiment 3:

[0072] The processing and calculation of the image data include the following steps:

[0073] S201. Feature Extraction: Extract key features that can represent the instrument packaging operation steps from a large amount of image data collected by the visual sensor group. These features cover multiple aspects. For example, the shape features of the instrument can help determine whether the used instrument is correct; the position features of the instrument can reflect whether it is in the appropriate operating position; and the posture features of the operation actions can reflect the standardization of the operation. When extracting features, advanced image recognition algorithms such as the Convolutional Neural Network (CNN) algorithm are used. CNN has strong feature learning ability and can automatically learn features related to the operation steps from the image data. By training on a large number of images containing different operation steps, CNN can accurately identify the feature patterns of various operation steps, providing strong support for subsequent step matching and standardization judgment.

[0074] S202. Step Matching: Use the template matching algorithm to match the extracted features with the preset standard operation step template. The standard operation step template is formulated based on industry specifications and actual operation experience and contains the standard feature descriptions of each operation step. The template matching algorithm judges the matching degree of the current operation step with the standard operation step by calculating the similarity between the extracted features and the standard template. For example, calculate the similarity scores of features such as the shape, position, and operation action posture of the instrument in the image with the corresponding features in the standard template. If the similarity score is high, it indicates that the current operation step is relatively well-matched with the standard step; conversely, if the similarity score is low, there may be non-standard operation situations.

[0075] S203. Standardization Judgment: Judge whether the operation meets the standard according to the matching result, and compare the execution of the operation steps in the image data within the same time interval to obtain the standardization difference. When performing the standardization judgment, trigger the rule engine in the growth detection unit. The rule engine is an intelligent judgment system based on preset rules and logics, and it analyzes according to the matching result and the preset rules. For example, if in the continuous image data, it is found that the execution order of the operation steps is inconsistent with the standard process, or the posture of the operation action is significantly different from the standard template, the rule engine will determine that the operation does not meet the standard and mark the images of the non-standard operation steps. At the same time, by comparing the execution of the operation steps within the same time interval, the standardization difference of the operation can be quantified, providing more accurate data for subsequent evaluation.

[0076] S204. Estimation of operation speed: The operation speed of instrument packing is obtained by calculating the number of operation steps completed between adjacent time points and dividing it by the time interval. For example, within a 10-second time interval, if the system detects that 5 operation steps have been completed, then the operation speed is 5÷10 = 0.5 steps / second. The operation speed is an important indicator for evaluating the packing operation efficiency. By monitoring the operation speed, the change in the work efficiency of the operator can be detected in a timely manner. If the operation speed is too fast, it may lead to non-standard operations and affect the packing quality; if the operation speed is too slow, the production efficiency will be reduced. The system can issue a warning for abnormal operation speed according to the preset operation speed threshold, so as to take measures for adjustment in a timely manner.

[0077] Example 4:

[0078] This example focuses on introducing the specific method for calculating the difference parameters, providing key data for evaluating the operation stability by quantifying the differences between the data obtained by different detection devices.

[0079] Calculate the difference parameters of the environmental data in the same partition obtained by two detection devices. The difference parameters include the position difference C wz , operation force difference C cl , and image similarity difference C xt . The calculation method of the position difference C wz is C wz = |the instrument position W zi obtained by the detection device in the area - the instrument position W z obtained by the scheduled detection device|. In practical applications, when the detection device performs multi-point scheduling, there will be measurement data of the instrument position in the same partition at two different time points or by different detection devices. For example, at a certain moment, the detection device in the area measures the position coordinates of the instrument as (x1, y1, z1), and after scheduling, another detection device measures the position coordinates of the instrument as (x2, y2, z2), then the position difference C wz is (assuming the position data is in the form of three-dimensional coordinates here).

[0080] For other differences in the difference parameters, such as the operation force difference C cl and the image similarity difference C xt , they are also calculated and obtained in a similar way to the position difference C wz . The operation force difference C cl is obtained by calculating the absolute value of the difference between the operation force values collected by the detection device in the area and the operation force values collected by the scheduled detection device. For example, if the operation force collected by the detection device in the area is F1 and the operation force collected by the scheduled detection device is F2, then the operation force difference C cl = |F1 - F2|.

[0081] Poor image similarity C xt The calculation of is relatively complex. It is necessary to first extract the features of the images collected by different detection devices, and then calculate the differences between them through specific image similarity calculation algorithms. Commonly used image similarity calculation methods include pixel-based methods, feature-based methods, etc. For example, when using the feature-based method, first extract the key features of the image, such as edge features, texture features, etc., and then calculate the similarity scores between these features. The difference between the similarity scores of the two images is used as the image similarity difference C xt . By calculating these difference parameters, the differences between the data obtained by different detection devices can be comprehensively quantified, providing an accurate data basis for generating the subsequent operation stability evaluation value.

[0082] Example 5:

[0083] This example details the process of generating the operation stability evaluation value and the specific method of executing corresponding strategies according to the evaluation results, aiming to achieve the evaluation of operation stability and the effective handling of abnormal situations.

[0084] The steps to generate the operation stability evaluation value W for the corresponding partition are as follows: czzi as follows:

[0085] S301. Normalize the difference parameters: Since the position difference C wz , the operation force difference C cl , the image similarity difference C xt and other difference parameters may have different dimensions and value ranges, directly using these parameters for calculation will affect the accuracy and comparability of the evaluation results. Therefore, it is necessary to normalize them. Normalization is to map each difference parameter to a unified value range. Commonly used normalization methods include min-max normalization, Z-score normalization, etc. Taking min-max normalization as an example, for the position difference C wz , assuming its minimum value within a period of time is and the maximum value is , then the normalized position difference . By normalizing all difference parameters, the influence of dimensions is eliminated, making them have the same weight and comparability in subsequent calculations.

[0086] S302. Build a data analysis model: According to the formula W czzi = b1C wz + b2C cl + b3C xt + H to generate the operation stability evaluation value W for the corresponding partition czzi . Among them, b1, b2, and b3 are the position difference C wz , the operation force difference C cl, the difference in image similarity C xt The proportionality coefficients, and b1 > b2 > b3 > 0. This indicates that when evaluating the operation stability, the position difference has the greatest impact on the evaluation result because the stability of the instrument position is directly related to the accuracy and safety of the packing operation; the operation force difference is the second, and the appropriate operation force is the key factor to ensure the packing quality; the difference in image similarity is relatively small, but it can also reflect the consistency and standardization of the operation actions. H is a constant correction coefficient used to fine-tune the calculation result to improve the accuracy of the evaluation. For example, in the evaluation of a certain partition, after normalization, the position difference The operation force difference The difference in image similarity Assume b1 = 0.5, b2 = 0.3, b3 = 0.2, and H = 0.1, then the operation stability evaluation value W czzi = 0.5×0.3 + 0.3×0.2 + 0.2×0.1 + 0.1 = 0.27.

[0087] Compare the operation stability evaluation value W czzi with the preset evaluation threshold, and execute the corresponding strategy according to the comparison result:

[0088] If the operation stability evaluation value W czzi does not exceed the evaluation threshold, it means that the operation in the current partition is relatively stable, and the system does not make a response action. For example, the preset evaluation threshold is 0.3. When the calculated W czzi = 0.25, the system considers the operation to be in a stable state and no additional intervention is required.

[0089] If the operation stability evaluation value W czzi exceeds the evaluation threshold, an alarm will be processed and an adjustment strategy will be executed. The alarm processing is carried out by setting an alarm to give an audible and visual alarm to attract the attention of relevant personnel. The adjustment strategy includes checking and calibrating the packing tools in the corresponding partition, and reminding the operator of the operation specifications. For example, when W czzi = 0.4 exceeds the evaluation threshold, the alarm gives an audible and visual alarm. At the same time, the system arranges professional personnel to check the packing tools to see if there are any problems such as damage or precision deviation, and calibrate the tools to ensure their normal operation. In addition, the system will send a reminder message of the operation specifications to the operator, such as displaying the correct operation steps and precautions on the display screen, or guiding the operator to adjust the operation method by voice prompt to improve the operation stability and packing quality.

[0090] Example 6:

[0091] In this embodiment, taking an instrument packing table with 4 partitions as an example, it is described in detail how the monitoring robot of the instrument packing table based on the AI image recognition function can issue timely reminders for problems such as over-placement, under-placement, incorrect model placement, and omission of indicators of instruments. Specifically, it includes:

[0092] Data collection: In each partition, detection devices are configured. The vision sensor group consists of a high-definition camera and a depth camera, and is integrally installed on the electric control track above the packing table. The high-definition camera is responsible for collecting detailed information such as the position image and operation action image of the instruments in the corresponding partition, and the depth camera assists in obtaining the spatial position information of the instruments. For example, when packing surgical instruments, the high-definition camera can clearly capture the shape and appearance characteristics of the instruments, and the depth camera can determine the specific position of the instruments on the packing table, accurate to the millimeter level. The position sensors are installed on the electric control tracks arranged around the packing table, and monitor the instrument positions in real time from horizontal and vertical directions to ensure full-range tracking of the changes in the instrument positions. The operation force sensor is connected to the outer electric control track through an L-shaped metal rod, and its sensing end is always in contact with the packing tool to collect the operation force magnitude during the packing operation in real time. For example, when tightening the packing belt, it can accurately measure the operation force value. The data collection unit continuously collects the environmental data of each partition to provide a basis for subsequent analysis.

[0093] Data preprocessing: After obtaining the environmental data of each partition, it enters the preprocessing unit. First, the environmental data is cleaned, and using a preset data cleaning algorithm, fuzzy data, error data, and redundant data are removed. For example, if a certain frame of image collected by the vision sensor group is blurred due to light interference, it is removed after being judged by the algorithm; the incorrect position data generated by the position sensor due to signal interference is also eliminated. Then, the environmental data in the same partition is aggregated to form an environmental data set corresponding to the partition, which is convenient for subsequent processing and analysis.

[0094] Determination and warning: The determination and warning unit compares each data in the environmental data with the preset standard range value of the corresponding partition. For the number of instruments, according to the surgical instrument packaging standard, the types and quantities of instruments that should be included in each surgical instrument package are preset. Taking the common debridement surgical instrument package as an example, the standard configuration is 3 scalpels, 2 tweezers, 2 scissors, etc. If the image collected by the visual sensor group is processed and analyzed, it is identified that the number of scalpels packaged in a certain partition is greater than 3, that is, it is determined that the instruments are placed too much; if it is less than 3, it is determined that the instruments are placed too little, and the determination and warning unit sends a warning signal at this time. For the instrument model, the image recognition technology is used to identify the appearance characteristics of the instrument and compare it with the preset standard instrument model image library. If the model of a pair of scissors is identified to be inconsistent with the model specified in the standard library, it is determined that the instrument model is misplaced and a warning is issued. For the sterilization indicator, image recognition is also used to determine whether it exists. If the corresponding item is not detected at the location where the sterilization indicator should be placed, it is determined that the sterilization indicator is missed and a warning is triggered. The warning method is to issue an alarm through an audible and visual alarm set near the packaging table to remind the operator to deal with it in time.

[0095] Operation detection: The operation detection unit extracts the image data collected by the visual sensor group. First, feature extraction is performed, and the deep learning algorithm is used to extract features representing the instrument packaging operation steps from the image data, such as feature vectors such as instrument grabbing action and placement action. Then, the extracted features are matched with the preset standard operation step template using the template matching algorithm. During the matching process, if it is found that a certain operation step is significantly different from the standard template, for example, the order of placing the instrument does not conform to the standard process, the operation is judged to be non-compliant according to the matching result. The execution of the operation steps in the image data is compared within the same time interval to obtain the normative difference, and the rule engine in the growth detection unit is triggered to mark the images that do not conform to the standard operation steps for subsequent analysis and processing. At the same time, the operation speed of instrument packaging is obtained by calculating the number of operation steps completed between adjacent time points and dividing it by the time interval. If the operation speed is too fast or too slow, it can also be used as a reference factor for judging whether the operation is standardized.

[0096] Multi-point scheduling: It has the following scheduling strategies: Strategy 1: When the packaging operation of a certain partition meets the requirements, the multi-point scheduling unit continues to keep the detection equipment of the partition stationary and continues to monitor; Strategy 2: When the packaging operation of a certain partition meets the requirements, the multi-point scheduling unit drives the detection equipment of the partition to the partition with the image mark of the non-compliant operation steps. For example, if the operation of partition No. 1 is in compliance with the regulations, and there is a problem of multiple equipment placement in partition No. 3 and the image has been marked, the detection equipment of partition No. 1 will be moved along the electric control track to partition No. 3, and the environmental data in partition No. 3 will be obtained again to further analyze the problem and check whether the operator has rectified the problem or whether there are other potential problems.

[0097] Evaluation and analysis: Calculate the difference parameters of the environmental data obtained by two detection devices in the same partition, including position difference, operating force difference, and image similarity difference. Taking the position difference as an example, assume that the position of the instrument obtained by the detection device in this area is W zi , and the position of the instrument obtained by the scheduled detection device is W z , and the position difference C wz = |W zi - W z |. Other differences are calculated in a similar manner. After normalizing the difference parameters, build a data analysis model. According to the formula W czzi = b1C wz + b2C cl + b3C xt + H to generate the operation stability evaluation value W czzi of the corresponding partition. Among them, b1, b2, and b3 are the proportionality coefficients of the position difference, operating force difference, and image similarity difference respectively, and b1 > b2 > b3 > 0, and H is a constant correction coefficient. Compare the operation stability evaluation value W czzi with the preset evaluation threshold. If it does not exceed the evaluation threshold, it indicates that the operation in this partition is relatively stable and no response action is taken; if it exceeds the evaluation threshold, an alarm is processed, an alarm is issued through an audible and visual alarm, and an adjustment strategy is executed. The adjustment strategy is to check and calibrate the packing tool corresponding to the partition, check whether the packing tool has a fault affecting the operation stability, and at the same time remind the operator of the operation specifications, such as displaying the correct operation process and precautions through the display screen to ensure the accuracy and stability of subsequent packing operations.

[0098] Embodiment 7

[0099] Based on the applications of the above various embodiments, this embodiment can also record, count, and store the instrument data packed by the robot , including the packing time, images, etc., so as to constitute a corresponding configured atlas of the packed instruments (stored in the form of a structured database, saving the packing data corresponding to each packed instrument: packing time, image, parameter configuration, etc.).

[0100] When the packing data needs to be queried:

[0101] The user / administrator can input the corresponding SQL retrieval instruction (including information such as the retrieval conditions and fields of the packed instrument to be consulted) into the database, retrieve the corresponding configured atlas of the archived instruments. After being retrieved and output by the database, it can be transmitted by a switch or other network to a video projection device (such as a projector), so as to project, play, and display it. The retrieval and projection function of the archived configured atlas of the instruments is realized, which is convenient for the user to view the instrument packing data.

[0102] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0103] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A monitoring robot for instrument packaging station based on AI image recognition function, characterized in that: include: The data acquisition unit detects the partitions of the instrument packaging table and uses the detection equipment to obtain the environmental data of each partition. The detection equipment is configured in each partition and includes a visual sensor group, a position sensor and an operating force sensor; A preprocessing unit, which preprocesses the environmental data obtained from each partition; The judgment warning unit compares each data in the environmental data with the preset standard range value of the corresponding partition. If the corresponding data is within the corresponding standard range value, no response action is taken. Otherwise, a warning is issued; The operation detection unit extracts the image data collected from the visual sensor group, processes and calculates the image data to obtain the standardization and operation speed of the corresponding partition instrument packaging operation. The process of processing and calculating the image data includes identifying the instrument packaging operation steps. When the steps are identified, the rule engine in the growth detection unit is triggered to mark the images that do not meet the standard operation steps. The multi-point dispatching unit, under the condition that the packaging operation of a certain partition meets the requirements, drives the detection equipment of the partition to the partition where the image mark of the operation step that does not meet the standard exists, so as to obtain the environmental data in the partition where the image mark of the operation step that does not meet the standard exists again; The evaluation and analysis unit calculates and obtains the difference parameters of the environmental data in the same partition obtained by two detection devices, and builds a data analysis model based on the difference parameters to generate the operation stability evaluation value W of the corresponding partition czzi , and the operational stability evaluation value W czzi Compare with the preset evaluation threshold and execute the corresponding strategy based on the comparison result.

2. The instrument packing station monitoring robot based on AI image recognition function according to claim 1 is characterized by: The visual sensor group is composed of a high-definition camera and a depth camera, and the visual sensor group is an integral structure and is arranged on an electric control track above the packaging table. The visual sensor group is used to collect position images and operation action images of equipment in the corresponding partition in the environmental data. The i in the partition environmental data obtained by the detection equipment represents the partition number, and the partition number is marked in a clockwise order, i=1, 2, ...n, where n is a positive integer.

3. The instrument packing station monitoring robot based on AI image recognition function according to claim 2 is characterized in that: The position sensor is arranged on the electric control track arranged around the packing table to monitor the position of the equipment in the corresponding partition in real time from the horizontal and vertical directions.

4. The instrument packing station monitoring robot based on AI image recognition function according to claim 3 is characterized by: The sensing end of the operating force sensor is always in contact with the packaging tool and is connected to the peripherally configured electric control track through an L-shaped metal rod. The operating force sensor is used to collect the operating force size during the packaging operation in the corresponding partition in the environmental data.

5. The instrument packing station monitoring robot based on AI image recognition function according to claim 4 is characterized in that: The steps for preprocessing the environmental data obtained for each partition are as follows: S101. Clean the environmental data to remove fuzzy data, erroneous data and redundant data; S102: Aggregate the environmental data in the same partition to form an environmental data set of the corresponding partition.

6. The instrument packing station monitoring robot based on AI image recognition function according to claim 5 is characterized in that: The steps for processing and calculating image data are as follows: S201, feature extraction: extracting features representing the instrument packaging operation steps from the image data; S202, step matching: using a template matching algorithm to match the extracted features with a preset standard operation step template; S203, normative judgment: judging whether the operation complies with the norm according to the matching result, comparing the execution of the operation steps in the image data within the same time interval, obtaining normative differences, and triggering the rule engine in the growth detection unit when making normative judgment; S204, operation speed estimation: the operation speed of device packaging is obtained by calculating the number of operation steps completed between adjacent time points and dividing the number by the time interval.

7. The instrument packing station monitoring robot based on AI image recognition function according to claim 6 is characterized by: Calculate and obtain the difference parameters of the environmental data in the same partition obtained by two detection devices, where the difference parameters include the position difference C wz , Operation force difference C cl , image similarity difference C xt , where the position difference C wz =|The device position W obtained by the detection equipment in this area zi -The device position W obtained by the scheduled detection equipment z |, for other differences in the difference parameter, the position difference C wz Calculate the acquisition in the same way.

8. The instrument packing station monitoring robot based on AI image recognition function according to claim 7 is characterized in that: Generate the operational stability evaluation value W of the corresponding partition czzi The steps are as follows: S301, position difference C in difference parameter wz , Operation force difference C cl , image similarity difference C xt Perform normalization processing; S302: Build a data analysis model to generate an operational stability evaluation value W for the corresponding partition czzi , the formula is as follows: W czzi =b1C wz +b2C cl +b3C xt +H Where b1, b2, and b3 are the position differences C wz , Operation force difference C cl , image similarity difference C xt The proportional coefficient is b1>b2>b3>0, and H is the constant correction coefficient.

9. The instrument packing station monitoring robot based on AI image recognition function according to claim 8 is characterized in that: The operational stability evaluation value W czzi The results of the comparison with the preset evaluation thresholds include: If the operational stability evaluation value W czzi If the evaluation threshold is not exceeded, no response action will be taken. If the operation is stable, the evaluation value W czzi If the assessment threshold is exceeded, an alarm is processed and an adjustment strategy is executed. The alarm is processed by setting an alarm to give an audible and visual alarm, which is used to prompt the execution of the adjustment strategy. The adjustment strategy is to check and calibrate the packaging tools of the corresponding partition and remind the operator of the operating specifications.

10. A monitoring method applied to the robot according to any one of claims 1 to 9, characterized in that: The following steps are involved: Data collection: Use the data collection unit to collect environmental data from each partition of the instrument packaging station; Data preprocessing: Clean and summarize the collected environmental data through the preprocessing unit; Determination and warning: The determination and warning unit compares the environmental data with the preset standard range value, and issues a warning if it exceeds the range; Operation detection: The operation detection unit processes and calculates the image data collected by the visual sensor group, identifies the operation norms and marks the images that do not meet the norms; Multi-point dispatching: When the partition operation compliance meets the requirements, the detection equipment will be dispatched to the marked partition; Evaluation and analysis: Calculate the difference parameters, build a data analysis model to generate an operation stability evaluation value, and then execute the corresponding strategy after comparing it with the evaluation threshold.

Images