Instrument package counting and checking method fusing RFID and visual identification

Abstract

The application relates to the technical field of instrument management, and specifically discloses an instrument bag counting and checking method combining RFID and visual identification, which comprises the following steps: firstly, collecting RFID tag signals and visual images of instruments, pre-processing the signals, and sorting the signals according to a sorting rule to obtain an RFID tag sequence and a visual sequence; then, adopting a dynamic time warping algorithm with a discard penalty to align and match the RFID tag sequence with the visual sequence, solving an optimal matching path, obtaining matching items, RFID tag sequence abnormal items and visual sequence abnormal items, and performing secondary screening on the RFID tag sequence abnormal items and the visual sequence abnormal items, so that the screened candidate matching items and the matching items are included in an instrument list; finally, checking the instrument list against a standard list, and outputting a counting and checking result. Through the steps of solving the optimal matching path and the like, the instrument can be counted and checked, the counting accuracy under complex working conditions can be improved, and the false alarm rate can be reduced.

Description

Technical Field

[0001] This invention relates to the field of medical device management technology, and in particular to a method for checking and verifying medical device packages that integrates RFID and visual recognition. Background Technology

[0002] With the rapid development of smart healthcare and refined operating room management, the traceability and inventory verification of surgical instruments throughout the entire process has become a core aspect of ensuring surgical safety. Traditional manual inventory methods are inefficient, subjective, and susceptible to fatigue and unexpected situations, resulting in high rates of false positives and false negatives, making them unsuitable for the management needs of modern operating rooms. Therefore, the industry is gradually adopting intelligent technologies such as machine vision and RFID (Radio Frequency Identification). Visual recognition enables the inspection of instrument appearance and quantity statistics, while RFID technology enables instrument identification and traceability. Today, multimodal fusion inventory technology has become the mainstream development direction for intelligent management of surgical instruments.

[0003] Currently, existing technologies have developed surgical instrument pack counting and verification solutions that integrate RFID and visual recognition. These solutions primarily rely on a simple comparison of visual image recognition and RFID radio frequency reading results to count the instruments. Existing technologies mainly use visual image detection to identify instruments on the worktable and count their quantity and category. Simultaneously, RFID readers read the quantity and category information from instrument tags. A simple numerical comparison is then performed between the visual recognition results and the RFID reading results. The consistency of these two types of data completes the instrument pack counting and verification, achieving a preliminary intelligent replacement for manual counting methods.

[0004] However, existing RFID and vision-based instrument inventory verification solutions still have technical shortcomings and cannot adapt to the complex real-world conditions of surgical instruments, such as metallic reflections, stacking obstructions, and radio frequency multipath interference. Current solutions only perform simple comparisons of quantity and category, without constructing a physical spatial coordinate system or aligning the spatial dimensions of the two modalities. This fails to address issues such as visual missed detections, RFID spurious signals, and modal data mismatches caused by instrument obstruction and misalignment, resulting in a high probability of false alarms. Existing fusion matching algorithms often employ fixed-weight cost calculation methods, which cannot adaptively adjust constraint weights based on environmental interference. Under complex conditions, matching accuracy drops significantly, and robustness is poor. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method for inventorying and verifying medical device packages that integrates RFID and visual recognition, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for inventorying and verifying medical device packages integrating RFID and visual recognition, comprising the following steps: When the instrument to be counted is detected to be placed in the counting area, the RFID tag signal and visual image of the instrument in the area are collected simultaneously. The RFID tag signal and the visual image are preprocessed to generate an RFID tag sequence and a visual sequence according to a sorting rule; The RFID tag sequence and the visual sequence are aligned and matched using a dynamic time warping algorithm with a drop penalty. The optimal matching path is solved, and the matching items, RFID tag sequence anomalies, and visual sequence anomalies in the optimal matching path are obtained. All matching items are summarized and a medical device list is generated. The abnormal items of the RFID tag sequence and the abnormal items of the visual sequence marker are further filtered, and the selected candidate matching items are added to the medical device list. The instrument list is compared with the preset standard list, and the final inventory and verification results are output.

[0007] Preferably, the preprocessing of the RFID tag signal and the visual image includes: The RFID tag signal is acquired, wherein the RFID tag signal includes tag code, radio frequency signal strength, and I / Q quadrature components; The phase difference angle of arrival of the I / Q quadrature components is calculated and the reference tag is calibrated to obtain the RFID physical coordinates of each tag signal; The radio frequency signal strength is analyzed to obtain the tag confidence level of each tag signal; The visual image is acquired, and instance segmentation is performed on the visual image to obtain the instance mask, pixel coordinates, class code, and class confidence of each instance in the image. The pixel coordinates are converted into physical coordinates using a transformation matrix calibrated by the camera to obtain visual physical coordinates; Feature extraction is performed on the visual image to obtain the geometric contour, physical size, and quality defects of each instance in the image.

[0008] Preferably, the sorting rule is as follows: Using the top left corner of the identification area as the origin of the physical coordinates, the tags are sorted in ascending order of horizontal coordinates and in ascending order of vertical coordinates when horizontal coordinates are the same, to obtain the RFID tag sequence and the visual sequence. The RFID tag sequence includes RFID physical coordinates, tag code, tag confidence level, and preset instrument standard information. The visual sequence includes visual physical coordinates, category code, category confidence level, geometric contour, physical dimensions, and quality defects.

[0009] Preferably, the content for aligning and matching the RFID tag sequence with the visual sequence includes: Traverse every element in the RFID tag sequence and visual sequence, and construct a matching cost matrix with the spatial positional deviation, category matching degree and signal confidence degree of the RFID tag sequence and visual sequence as constraint factors. The matching cost is calculated using the following formula: ; in, The spatial position deviation is calculated from the normalized Euclidean distance between the RFID physical coordinates and the visual physical coordinates. Category matching degree is obtained from the consistency between label code and category code. The signal confidence score is calculated jointly from the label confidence score and the category confidence score. For the i-th RFID tag sequence, For the j-th visual sequence, This is the weighting coefficient for spatial location deviation. This is a weighting coefficient for category matching. is the weighting coefficient for the signal confidence, and ; Set the discard penalty value and cost threshold, construct a recursive formula that supports element discarding operations, and solve for the optimal matching path with the minimum total cost along the matching cost matrix in the forward direction; The recursive formula is: ; in, To discard the penalty value, For cumulative matching costs; By tracing back the optimal matching path, we obtain matching items, RFID tag sequence anomalies, and visual sequence anomalies.

[0010] Preferably, the weighting coefficients of spatial location deviation, category matching degree, and signal confidence degree in the matching cost calculation formula are dynamically adjusted according to the environment of the inventory area: Real-time acquisition of current environmental interference parameters, including the degree of metal interference and the proportion of image reflection, and adjustment of the weight coefficients of spatial position deviation, category matching degree and signal confidence degree according to the preset environment-weight mapping relationship; The environment-weight mapping relationship is a piecewise linear function. The piecewise linear function takes the environmental interference parameter as the independent variable and the weight coefficient as the dependent variable, and divides the range of values ​​of the environmental interference parameter into multiple continuous sub-intervals. In each sub-interval, the weight coefficient and the environmental interference parameter have a linear relationship.

[0011] Preferably, the secondary screening includes: The visual instances in the visual sequence anomalies are reread using RFID. If the reread is successful, the instance is removed from the visual sequence anomalies and marked as a candidate match. The tag signals in the RFID tag sequence anomaly items are screened for authenticity. If a tag signal is determined to be fake, it is removed from the RFID tag sequence anomaly items.

[0012] Preferably, the content of the secondary supplementary reading includes: Obtain the visual physical coordinates of visual instances of visual anomalies in the visual sequence, and delineate the scanning area based on the visual physical coordinates; Enhance RFID reading parameters, perform RFID scanning within the scanning area, obtain local RFID tag signals, and preprocess the signals; Calculate the matching cost between local RFID tags and visual instances of visual sequence anomalies. If the matching cost is less than the cost threshold, establish a mapping relationship, mark it as a candidate match, and remove it from the visual sequence anomalies. Mark the remaining visual sequence anomalies as "label dropped" and output a review instruction.

[0013] Preferably, the content of the authenticity screening includes: Obtain the radio frequency signal strength and I / Q quadrature components of the RFID tag sequence anomaly signal, and calculate the radio frequency signal fluctuation amplitude and phase standard deviation; When the fluctuation amplitude of the radio frequency signal is greater than the fluctuation threshold and the phase standard deviation is greater than the phase threshold, the signal is determined to be a fake signal and removed from the abnormal items of the RFID tag sequence. Mark the remaining RFID tag sequence anomalies as visual occlusions and output a verification command.

[0014] As described above, the instrument package counting and verification method integrating RFID and visual recognition provided by the present invention has at least the following beneficial effects: This invention provides a method for inventorying and verifying medical device packages that integrates RFID and visual recognition. By constructing a unified physical coordinate system for workstations, RFID tags and visual instances are mapped to the same spatial reference. Combined with a dynamic time warping algorithm with a discard penalty, the method achieves globally optimal alignment of ordered sequences, solving the problems of misalignment, mismatch, and missed or false detection of medical devices. At the same time, the element discarding mechanism can adapt to real working conditions of occlusion and missed reading, significantly improving the accuracy of dual-modal pairing and the reliability of inventory results.

[0015] By collecting environmental interference parameters in real time and the preset piecewise linear mapping relationship, the weight ratio of three constraint factors—spatial location, category matching, and signal confidence—is dynamically adjusted. This overcomes the defect of existing fixed-weight schemes where matching accuracy drops sharply in metal and mirror reflection scenarios. It can be adapted to the inventory conditions of different departments such as orthopedics, general surgery, and microsurgical instruments, and the algorithm has stronger environmental adaptability. Attached Figure Description

[0016] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for checking and verifying medical device packages that integrates RFID and visual recognition, according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, the present invention provides a method for inventorying and verifying medical device packages that integrates RFID and visual recognition, comprising the following steps: S1. When the instrument to be counted is detected to be placed in the counting area, the RFID tag signal and visual image of the instrument in the area are collected simultaneously. In this embodiment, when the workstation sensor detects that the medical device package to be counted is placed within the counting and identification area, the system synchronously triggers the RFID reading and writing module and the image acquisition module to ensure that the spatiotemporal reference of the two types of data is consistent. The RFID reading and writing module uses an ultra-high frequency RFID reader that supports baseband I / Q quadrature component output and real-time RSSI reporting, equipped with a 2×2 planar antenna array. The clocks of each antenna channel are synchronized to ensure strict alignment of sampling times. The counting and identification area is continuously polled to collect and acquire RFID tag signals. The RFID tag signal includes the unique EPC code of each tag, the received signal strength of each channel, and the I / Q quadrature components of each antenna channel. The visual image is acquired by an industrial area array camera at the top of the workstation, obtaining an RGB image of the entire counting area.

[0020] S2. Preprocess the RFID tag signal and the visual image to generate an RFID tag sequence and a visual sequence according to the sorting rules; The preprocessing of the RFID tag signal and the visual image includes: The RFID tag signal is acquired, wherein the RFID tag signal includes tag code, radio frequency signal strength, and I / Q quadrature components; The phase difference angle of arrival of the I / Q quadrature components is calculated and the reference tag is calibrated to obtain the RFID physical coordinates of each tag signal; The radio frequency signal strength is analyzed to obtain the tag confidence level of each tag signal; The visual image is acquired, and instance segmentation is performed on the visual image to obtain the instance mask, pixel coordinates, class code, and class confidence of each instance in the image. The pixel coordinates are converted into physical coordinates using a transformation matrix calibrated by the camera to obtain visual physical coordinates; Feature extraction is performed on the visual image to obtain the geometric contour, physical size, and quality defects of each instance in the image.

[0021] The sorting rules are as follows: Using the top left corner of the identification area as the origin of the physical coordinates, the tags are sorted in ascending order of horizontal coordinates and in ascending order of vertical coordinates when horizontal coordinates are the same, to obtain the RFID tag sequence and the visual sequence. The RFID tag sequence includes RFID physical coordinates, tag code, tag confidence level, and preset instrument standard information. The visual sequence includes visual physical coordinates, category code, category confidence level, geometric contour, physical dimensions, and quality defects.

[0022] In this embodiment, after receiving the RFID tag signal and the visual image, preprocessing is performed on both. The RFID tag signal preprocessing steps are as follows: First, signal cleaning and deduplication are performed. The RFID tag signals are grouped according to the RFID tag, and data of the same tag are grouped together. Occasional signals with fewer than 3 reads are removed, and the average value of multiple groups of sampled data is taken. Next, the I / Q quadrature components of each tag are substituted into the arctangent function to obtain the carrier phase. Using the array reference antenna as a reference, the phase difference between the X and Y direction baselines is calculated. Combining the RF carrier wavelength and the antenna baseline spacing, the signal angle of arrival is solved. Based on the relationship between the antenna installation height and the tangent of the angle of arrival, the initial physical coordinates of the tag are calculated. Reference tags with known coordinates are preset at the four corners of the workstation. The theoretical phase value without environmental interference is calculated. Before each inventory check, the real-time phase value of the reference tag is synchronously collected. The deviation between the phase measurement value and the theoretical value is calculated. Differential correction is used to compensate for environmental drift of the initial physical coordinates to obtain the RFID physical coordinates of the tag signal. Finally, the radio frequency signal strength is obtained, the mean signal strength is calculated and normalized to obtain the tag confidence level of each tag signal.

[0023] The visual image preprocessing steps are as follows: First, the original image is sequentially processed with polarization de-reflection, homomorphic filtering brightness equalization, and morphological denoising. Next, a lightweight instance segmentation network is input, outputting the instance mask, bounding box, center point pixel coordinates, device category code, and category confidence score for each device instance. The homography transformation matrix and lens distortion coefficients are obtained through offline Zhang's camera calibration. Using the mapping relationship from pixel coordinates to world coordinates, the center point pixel coordinates of each instance are converted to visual physical coordinates, sharing the same coordinate reference with the RFID side. The geometric contour of the device instance is extracted from the instance mask, and the extracted contour is smoothed and approximated with polygons. The physical dimensions of the device instance are calculated using a preset physical size ratio, while simultaneously identifying quality defects such as surface corrosion and edge defects.

[0024] Two ordered sequences are generated using completely consistent sorting rules: the first sorting rule uses the top left corner of the identification area as the origin of the physical coordinates and the horizontal coordinates of the target center point in ascending order; if the difference between the horizontal coordinates of two targets is less than or equal to 5mm, then the second sorting rule uses the vertical coordinates in ascending order. Finally, an RFID tag sequence and a visual sequence are generated. In the RFID tag sequence, each element includes RFID physical coordinates, tag code, tag confidence level, and instrument category code. In the generated visual sequence, each element includes visual physical coordinates, category code, category confidence level, geometric contour features, physical dimensions, and quality defect features.

[0025] S3. Use a dynamic time warping algorithm with a discard penalty to align and match the RFID tag sequence with the visual sequence, solve for the optimal matching path, and obtain the matching items, RFID tag sequence anomalies, and visual sequence marker anomalies in the optimal matching path. The content of aligning and matching the RFID tag sequence with the visual sequence includes: Traverse every element in the RFID tag sequence and visual sequence, and construct a matching cost matrix with the spatial positional deviation, category matching degree and signal confidence degree of the RFID tag sequence and visual sequence as constraint factors. The matching cost is calculated using the following formula: ; in, The spatial position deviation is calculated from the normalized Euclidean distance between the RFID physical coordinates and the visual physical coordinates. Category matching degree is obtained from the consistency between label code and category code. The signal confidence score is calculated jointly from the label confidence score and the category confidence score. For the i-th RFID tag sequence, For the j-th visual sequence, This is the weighting coefficient for spatial location deviation. This is a weighting coefficient for category matching. is the weighting coefficient for the signal confidence, and ; Set the discard penalty value and cost threshold, construct a recursive formula that supports element discarding operations, and solve for the optimal matching path with the minimum total cost along the matching cost matrix in the forward direction; The recursive formula is: ; in, To discard the penalty value, For cumulative matching costs; By tracing back the optimal matching path, we obtain matching items, RFID tag sequence anomalies, and visual sequence anomalies.

[0026] The weighting coefficients of spatial location deviation, category matching degree, and signal confidence in the matching cost calculation formula are dynamically adjusted according to the environment of the inventory area: Real-time acquisition of current environmental interference parameters, including the degree of metal interference and the proportion of image reflection, and adjustment of the weight coefficients of spatial position deviation, category matching degree and signal confidence degree according to the preset environment-weight mapping relationship; The environment-weight mapping relationship is a piecewise linear function. The piecewise linear function takes the environmental interference parameter as the independent variable and the weight coefficient as the dependent variable, and divides the range of values ​​of the environmental interference parameter into multiple continuous sub-intervals. In each sub-interval, the weight coefficient and the environmental interference parameter have a linear relationship.

[0027] In this embodiment, each element within the RFID tag sequence and visual sequence is traversed, and the spatial position deviation, category matching degree, and signal confidence degree of each group of elements are calculated. The Euclidean distance between the RFID physical coordinates and the visual physical coordinates is calculated and normalized using the maximum diagonal physical length of the identification area as a reference to obtain the spatial position deviation, which characterizes the degree of spatial overlap between the RFID tag and the visual instance in a unified physical coordinate system. The RFID tag code and the visual category code are obtained, and the code consistency is compared to obtain the category matching degree. If the codes match, it is recorded as 1; otherwise, it is recorded as 0. The category matching degree characterizes whether the instrument category bound to the RFID tag is consistent with the instrument category obtained from visual recognition. The joint confidence degree of the tag confidence degree and the category confidence degree is calculated using geometric mean, and the absolute value of the joint confidence degree minus 1 is calculated to obtain the signal confidence degree, which characterizes the overall reliability of the current pairing.

[0028] The weighting coefficients employ a dynamic adjustment mechanism to adapt to the current inventory environment in real time. The calculation steps are as follows: First, the environmental interference parameters of the current scene are calculated in real time, including the degree of metallic interference and the proportion of image reflection. The degree of metallic interference is obtained by normalizing the average phase standard deviation of all valid labels, and the proportion of image reflection is calculated by the ratio of highlight pixels to total pixels in the image. A preset environment-weight piecewise linear function is then called, using the environmental interference parameters as independent variables to output the corresponding weighting coefficients. This piecewise linear function is obtained through offline calibration.

[0029] The steps for obtaining the piecewise linear function corresponding to the degree of metal interference are as follows: First, N test points with known precise physical coordinates are evenly selected on the counting station table. Data is collected under different environmental conditions, including but not limited to environments with no metal interference, low metal interference, medium metal interference, high metal interference, and extremely strong metal interference. Under each condition, test tags are placed sequentially at each test point, and the RFID physical coordinates and visual physical coordinates are recorded respectively. The average error of the RFID physical coordinates and the average error of the visual physical coordinates are calculated under each condition. Then, the positioning reliability ratio of RFID relative to vision is calculated under each condition. The calculation formula is as follows: ,in The reliability ratio under the m-th operating condition Let m be the average error of the RFID physical coordinates under the m-th working condition. R represents the average error of the visual physical coordinates under the m-th working condition. When R > 0.5, it indicates that the accuracy of the RFID physical coordinates is better than that of the visual physical coordinates; when R < 0.5, it indicates that the accuracy of the visual coordinates is better than that of the RFID physical coordinates. Next, a scatter plot is drawn with the degree of metal interference as the x-axis and the reliability ratio as the y-axis. The piecewise slope between adjacent working condition points is calculated. When the absolute value of the difference between the slopes of adjacent segments exceeds a preset slope threshold, the degree of metal interference corresponding to that working condition point can be determined as a segment node. The slope threshold is the accuracy control parameter for piecewise linear fitting, and its general range is 0.2-1.0. Specifically, the degree of metal interference during operation can be set to 0, 0.25, 0.50, 0.75, and 1.00, respectively. Assuming the calculated corresponding reliability ratios are 0.60, 0.57, 0.44, 0.30, and 0.24, and the preset slope threshold is 0.3, the absolute values ​​of the segmented slope differences between the metal interference levels of 0.25 to 0.50 and 0.50 to 1.00 are greater than the preset slope threshold. Therefore, the operating points with metal interference levels of 0.25 and 0.75 are designated as segmentation nodes, resulting in three continuous sub-intervals: (0, 0.25), (0.25, 0.75), and (0.75, 1.00). Within each sub-interval, the spatial position deviation weight, category matching weight, and signal confidence weight are all linearly related to the degree of metal interference. The spatial position deviation weight is set to... ,in and The fitting parameters were obtained by least squares linear regression. Let m be the degree of metal interference in the m-th working condition. These are segmentation nodes. The signal confidence weight increases with the degree of metallic interference. in and These are the preset upper and lower weight limits, obtained through multi-condition matching accuracy testing. Specifically, the upper weight limit can be 0.25, and the lower weight limit can be 0.1. The category matching degree weight is determined by the normalization condition that the sum of the three weights is always equal to 1, and the calculation formula is as follows: ,in These are the weighting coefficients calculated from the spatial position deviation weighting expression under the same working conditions. These are the weighting coefficients calculated from the signal confidence weighting expression under the same operating condition. All three types of weighting coefficients are non-negative and satisfy the following conditions: Based on the obtained segmented nodes and their respective weight coefficients, a complete piecewise linear function is assembled. The piecewise linear function expression corresponding to the spatial location deviation weight is as follows: ,in For segmentation nodes, Let P be the slope of each interval, and P be the total number of segments.

[0030] The steps for obtaining the piecewise linear function corresponding to the image reflectivity ratio are the same as those for obtaining the degree of metal interference. Specifically, a standard test instrument sample set (containing 10 instruments of different materials and surface finishes) is placed on the inventory workstation table. The angle of the shadowless lamp is adjusted to obtain 5 levels of reflectivity conditions, with the illumination angles being 30°, 45°, 60°, 75°, and 90°. Under each condition, the same visual recognition algorithm is used to identify the test sample set, and the accuracy of visual category recognition and coordinate positioning error are statistically analyzed. A scatter plot is drawn with the image reflectivity ratio as the abscissa and the visual category recognition accuracy as the ordinate. The piecewise slope between adjacent condition points is calculated, and condition points with slope changes exceeding a preset threshold (e.g., 0.3) are identified as segment nodes. Then, the least squares method is used to fit the linear expression of the weight coefficients in each interval, assembling them into a complete piecewise linear function of the image reflectivity ratio.

[0031] The degree of metallic interference in the current measurement environment is obtained, and then substituted into the piecewise linear function expressions corresponding to the spatial position deviation weight, category matching degree weight, and signal confidence degree weight to obtain the three weights of the metallic interference degree in the current measurement environment. The image reflectance ratio in the current measurement environment is also obtained, and then substituted into the piecewise linear function expressions corresponding to the spatial position deviation weight, category matching degree weight, and signal confidence degree weight to obtain the three weights of the image reflectance ratio in the current measurement environment. The weighted summation of the two types of weights—metallic interference degree and image reflectance ratio—is used to obtain the weight coefficients of spatial position deviation, category matching degree, and signal confidence degree. The expression for the weighted summation of spatial position deviation is as follows: ,in The weighting coefficients corresponding to the degree of metal interference. The weighting coefficients corresponding to the proportion of image reflectivity. The spatial positional deviation is calculated using a piecewise linear function corresponding to the proportion of image reflectance. Specifically, , The weighted sum of category matching degree and signal confidence degree uses the same weight coefficient.

[0032] A discard penalty value and a cost threshold are set. The discard penalty value represents the cost of discarding a single sequence element, and its value matches the normalized range of the matching cost to balance the mismatch rate of the algorithm. The cost threshold is the confidence threshold for a single pairing. Pairings with a matching cost higher than the threshold are judged as high-risk mismatches and are forcibly broken up and classified as anomalies. The discard penalty value and cost threshold are determined by traversing and optimizing an offline labeled test set under multiple working conditions. The specific values ​​for the discard penalty value and cost threshold are 0.4 and 0.6, respectively.

[0033] Construct a dynamic programming recurrence relation that supports element discarding operations: The operations are categorized into three types: the first is pairing, which pairs the i-th RFID tag with the j-th visual sequence; the second is discarding RFID tags, which do not match the corresponding visual elements; and the third is discarding visual instances, which do not match the corresponding RFID tags. Starting from the matrix... To begin, calculate the minimum cumulative cost of all grid points row by row and column by column in a forward direction to obtain the complete matrix.

[0034] From the end of the matrix The optimal path is traced back in reverse, where M is the total number of RFID tag sequences and N is the total number of visual sequence instances. Based on the operation type at each step of the path, three types of results are obtained: Matching items: Pairs of elements in the path that have undergone pairing operations are determined to correspond to the same physical instrument; Visual sequence anomalies: Elements in the path that have undergone a visual discard operation, i.e., instrument instances that were only visually recognized but did not match the corresponding RFID tag; RFID tag sequence anomalies: Elements in the path that perform an RFID discard operation, i.e., tag signals that are only read by RFID but not matched with the corresponding visual instance.

[0035] At the same time, a fallback check is performed on all matching items. If the cost of a single matching group is greater than the cost threshold, the pairing is forcibly broken up and the corresponding elements are classified into two categories of abnormal items to avoid high-risk mismatches.

[0036] S4. Summarize all matching items and generate a medical device list. Perform secondary screening on the RFID tag sequence anomalies and visual sequence marker anomalies, and add the selected candidate matching items to the medical device list. The secondary screening includes: The visual instances in the visual sequence anomalies are reread using RFID. If the reread is successful, the instance is removed from the visual sequence anomalies and marked as a candidate match. The tag signals in the RFID tag sequence anomaly items are screened for authenticity. If a tag signal is determined to be fake, it is removed from the RFID tag sequence anomaly items.

[0037] The content of the second supplementary reading includes: Obtain the visual physical coordinates of visual instances of visual anomalies in the visual sequence, and delineate the scanning area based on the visual physical coordinates; Enhance RFID reading parameters, perform RFID scanning within the scanning area, obtain local RFID tag signals, and preprocess the signals; Calculate the matching cost between local RFID tags and visual instances of visual sequence anomalies. If the matching cost is less than the cost threshold, establish a mapping relationship, mark it as a candidate match, and remove it from the visual sequence anomalies. Mark the remaining visual sequence anomalies as "label dropped" and output a review instruction.

[0038] The content of the authenticity screening includes: Obtain the radio frequency signal strength and I / Q quadrature components of the RFID tag sequence anomaly signal, and calculate the radio frequency signal fluctuation amplitude and phase standard deviation; When the fluctuation amplitude of the radio frequency signal is greater than the fluctuation threshold and the phase standard deviation is greater than the phase threshold, the signal is determined to be a fake signal and removed from the abnormal items of the RFID tag sequence. Mark the remaining RFID tag sequence anomalies as visual occlusions and output a verification command.

[0039] In this embodiment, all verified matches are first aggregated to generate an initial medical device list. Then, a secondary screening is performed on two types of anomalies. The step of targeted secondary reading for visual sequence anomalies includes: for each visual sequence anomaly, extracting the visual physical coordinates of the anomaly instance; defining a 40mm × 40mm local focused scanning area centered on the coordinates; increasing the RFID reader's transmission power by 3-6dBm and extending the single acquisition window duration by 100ms; and performing RFID scanning within the scanning area to obtain the local RFID tag signal. Next, standardization preprocessing is performed on the local RFID tag signal, using the same operations as the first preprocessing to calculate the physical coordinates and confidence level. The matching cost between the anomaly instance and the local tag is calculated. If the matching cost is less than the cost threshold, a pairing mapping is established, the anomaly is marked as a candidate match, removed from the visual anomalies, and added to the medical device list. Instances without a matching tag after secondary reading are marked as tag detached, and a re-labeling verification instruction is output.

[0040] The steps for verifying the authenticity of RFID tag sequence anomalies include: extracting the RSSI, time-series data, and continuous phase sample values ​​of the RFID tag sequence anomalies, and calculating the radio frequency signal fluctuation amplitude and phase standard deviation. When the signal fluctuation amplitude exceeds a preset fluctuation threshold (e.g., 15 dBm) and the phase standard deviation exceeds a preset phase threshold (e.g., ... If a label does not meet the criteria for a false signal, it is determined to be a spurious signal generated by metal multipath reflection and is not included in the final inventory. Labels that do not meet the criteria for spurious signal determination are determined to be genuine instrument labels, marked as visual occlusion abnormalities, and an instruction is output to manually remove the occlusion for verification.

[0041] Among them, the preset fluctuation threshold and phase threshold are determined through offline calibration. The offline calibration steps are as follows: at the standard inventory station, the fluctuation amplitude and phase standard deviation of the radio frequency signal of the real label at the fixed position are collected, the fluctuation amplitude and phase standard deviation of the radio frequency signal of the typical pseudo signal sample generated by the reflection of the metal tray are collected, the distribution of the signal fluctuation amplitude and phase standard deviation of the two types of samples is statistically analyzed, and the value with the highest classification accuracy of the two types of samples is selected as the preset threshold.

[0042] After completing the second screening, the final list of medical devices is updated, which includes all normal matching items and candidate matching items.

[0043] S5. Compare the instrument list with the preset standard list and output the final inventory and verification results.

[0044] The steps for comparing the instrument list with the preset standard list and outputting the final inventory and verification results are as follows: The standard list includes the instrument category code, the corresponding quantity of each category, and the total number of instruments in the instrument package.

[0045] If the actual counted quantity of all categories is completely consistent with the quantity in the standard list, and the total number of instruments is also completely consistent, the count verification is deemed to have passed, the system unlocks the packaging process, and allows the instrument pack to enter the subsequent sterilization and packaging process.

[0046] If any category has a shortage or surplus in quantity, the current inventory verification will be deemed unsuccessful. The system will trigger an abnormal alarm and lock the packaging process, simultaneously displaying the difference in category, quantity, and corresponding location. The inventory verification can be re-initiated after the operator handles the situation on-site.

[0047] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0048] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for inventorying and verifying medical device packages integrating RFID and visual recognition, characterized in that, include: When the instrument to be counted is detected to be placed in the counting area, the RFID tag signal and visual image of the instrument in the area are collected simultaneously. The RFID tag signal and the visual image are preprocessed to generate an RFID tag sequence and a visual sequence according to a sorting rule; The RFID tag sequence and the visual sequence are aligned and matched using a dynamic time warping algorithm with a drop penalty. The optimal matching path is solved, and the matching items, RFID tag sequence anomalies, and visual sequence anomalies in the optimal matching path are obtained. All matching items are summarized and a medical device list is generated. The abnormal items of the RFID tag sequence and the abnormal items of the visual sequence marker are further filtered, and the selected candidate matching items are added to the medical device list. The instrument list is compared with the preset standard list, and the final inventory and verification results are output.

2. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 1, characterized in that, The preprocessing of the RFID tag signal and the visual image includes: The RFID tag signal is acquired, wherein the RFID tag signal includes tag code, radio frequency signal strength, and I / Q quadrature components; The phase difference angle of arrival of the I / Q quadrature components is calculated and the reference tag is calibrated to obtain the RFID physical coordinates of each tag signal; The radio frequency signal strength is analyzed to obtain the tag confidence level of each tag signal; The visual image is acquired, and instance segmentation is performed on the visual image to obtain the instance mask, pixel coordinates, class code, and class confidence of each instance in the image. The pixel coordinates are converted into physical coordinates using a transformation matrix calibrated by the camera to obtain visual physical coordinates; Feature extraction is performed on the visual image to obtain the geometric contour, physical size, and quality defects of each instance in the image.

3. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 1, characterized in that, The sorting rules are as follows: Using the top left corner of the identification area as the origin of the physical coordinates, the tags are sorted in ascending order of horizontal coordinates and in ascending order of vertical coordinates when horizontal coordinates are the same, to obtain the RFID tag sequence and the visual sequence. The RFID tag sequence includes RFID physical coordinates, tag code, tag confidence level, and preset instrument standard information. The visual sequence includes visual physical coordinates, category code, category confidence level, geometric contour, physical dimensions, and quality defects.

4. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 1, characterized in that, The content of aligning and matching the RFID tag sequence with the visual sequence includes: Traverse every element in the RFID tag sequence and visual sequence, and construct a matching cost matrix with the spatial positional deviation, category matching degree and signal confidence degree of the RFID tag sequence and visual sequence as constraint factors. The matching cost is calculated using the following formula: ； in, The spatial position deviation is calculated from the normalized Euclidean distance between the RFID physical coordinates and the visual physical coordinates. Category matching degree is obtained from the consistency between label code and category code. The signal confidence score is calculated jointly from the label confidence score and the category confidence score. For the i-th RFID tag sequence, For the j-th visual sequence, This is the weighting coefficient for spatial location deviation. This is a weighting coefficient for category matching. is the weighting coefficient for the signal confidence, and ; Set the discard penalty value and cost threshold, construct a recursive formula that supports element discarding operations, and solve for the optimal matching path with the minimum total cost along the matching cost matrix in the forward direction; The recursive formula is: ； in, To discard the penalty value, For cumulative matching costs; By tracing back the optimal matching path, we obtain matching items, RFID tag sequence anomalies, and visual sequence anomalies.

5. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 4, characterized in that, The weighting coefficients of spatial location deviation, category matching degree, and signal confidence in the matching cost calculation formula are dynamically adjusted according to the environment of the inventory area: Real-time acquisition of current environmental interference parameters, including the degree of metal interference and the proportion of image reflection, and adjustment of the weight coefficients of spatial position deviation, category matching degree and signal confidence degree according to the preset environment-weight mapping relationship; The environment-weight mapping relationship is a piecewise linear function. The piecewise linear function takes the environmental interference parameter as the independent variable and the weight coefficient as the dependent variable, and divides the range of values ​​of the environmental interference parameter into multiple continuous sub-intervals. In each sub-interval, the weight coefficient and the environmental interference parameter have a linear relationship.

6. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 1, characterized in that, The secondary screening includes: The visual instances in the visual sequence anomalies are reread using RFID. If the reread is successful, the instance is removed from the visual sequence anomalies and marked as a candidate match. The tag signals in the RFID tag sequence anomaly items are screened for authenticity. If a tag signal is determined to be fake, it is removed from the RFID tag sequence anomaly items.

7. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 6, characterized in that, The content of the second supplementary reading includes: Obtain the visual physical coordinates of visual instances of visual anomalies in the visual sequence, and delineate the scanning area based on the visual physical coordinates; Enhance RFID reading parameters, perform RFID scanning within the scanning area, obtain local RFID tag signals, and preprocess the signals; Calculate the matching cost between local RFID tags and visual instances of visual sequence anomalies. If the matching cost is less than the cost threshold, establish a mapping relationship, mark it as a candidate match, and remove it from the visual sequence anomalies. Mark the remaining visual sequence anomalies as "label dropped" and output a review instruction.

8. The method for inventorying and verifying medical device packages integrating RFID and visual recognition according to claim 6, characterized in that, The content of the authenticity screening includes: Obtain the radio frequency signal strength and I / Q quadrature components of the RFID tag sequence anomaly signal, and calculate the radio frequency signal fluctuation amplitude and phase standard deviation; When the fluctuation amplitude of the radio frequency signal is greater than the fluctuation threshold and the phase standard deviation is greater than the phase threshold, the signal is determined to be a fake signal and removed from the abnormal items of the RFID tag sequence. Mark the remaining RFID tag sequence anomalies as visual occlusions and output a verification command.

Images