A medical consumable batch in-out warehouse information intelligent registration method and system

Abstract

The application discloses a kind of medical consumables batch warehousing information intelligent registration method and system, obtain multiple medical consumables images by camera array, carry out OCR character recognition processing, select an image as the first image, map the remaining images to the first image according to homographic transformation relationship to obtain global image;Global image is input into image segmentation model, and is segmented into several packaging areas with specific identification, and is subjected to OCR character extraction processing respectively;Packaging surface text information is divided into static core information and dynamic core information, and multidimensional matching recognition is carried out in multi-level matching query library, to obtain the cumulative score of multidimensional matching, determine the key information for medical consumables warehousing registration.The present application aims to solve the problem that the existing medical consumables warehousing registration is not highly automated, is easily affected by information recognition interference, and the registration efficiency and accuracy are insufficient, greatly improves the automation and high precision of medical consumables in warehouse management.

Description

Technical Field

[0001] This invention belongs to the field of medical consumables inventory management technology, specifically relating to a method and system for intelligent registration of batch entry and exit information of medical consumables. Background Technology

[0002] The registration of medical consumables entering and leaving the warehouse is a crucial link between medical supply warehousing and supply chain management. It directly affects the accuracy of inventory records, the compliance of consumable circulation, and the timeliness and safety of consumable supply during clinical diagnosis and treatment. It is an indispensable and important component of the medical industry's quality control system. The industry commonly uses barcode or RFID tag reading to complete the registration process. By scanning the preset markings on the consumable packaging, basic product information can be quickly retrieved and entered, effectively simplifying the manual registration process and improving warehousing efficiency.

[0003] However, this combination of label scanning and manual registration has many limitations in the practical application of medical consumables warehousing. It is difficult to adapt to the special characteristics of medical consumables and the industry's regulatory requirements, and has the following drawbacks: First, there is a strong reliance on the integrity and standardization of barcodes or RFID tags. Medical consumables are diverse in type, and some consumables have thin, easily wrinkled packaging. Friction during handling and chemical corrosion during sterilization processes can easily cause barcodes to become soiled or detached, or RFID tags to become invalid and unreadable by scanning equipment. Once a tag becomes invalid, the information of each consumable must be manually checked and entered into the system, which not only significantly reduces the efficiency of inbound and outbound operations but also easily leads to inventory data deviations due to human error, posing risks to the supply and inventory management of clinical consumables.

[0004] Secondly, medical consumables have far higher requirements for the sophistication of traceability information than ordinary commodities, and the information carried by existing labels is insufficient to meet the needs of full-process traceability. Barcodes or RFID tags typically only record basic static information such as the name, specifications, and manufacturer of the consumables. However, core dynamic information in medical scenarios, such as production batch number, expiration date, sterilization date, and batch traceability code, is mostly printed in plain text in different locations on the packaging and cannot be directly extracted through standard labels. During warehousing, operators must manually search, verify, and enter this critical information, which is a cumbersome and error-prone process and cannot meet the stringent requirements of the medical industry for consumable expiration date control, batch traceability, and compliance auditing.

[0005] Furthermore, existing labeling is not sufficiently adaptable to the multiple packaging levels and specifications of medical consumables. Medical consumables encompass various forms, including large boxes, medium-sized individual packages, and small single packages. The packaging styles and information layouts of different categories of consumables vary significantly, and some niche consumables suffer from issues such as non-standard information printing and inconsistent fonts. There is a lack of a universal method that can adapt to multiple packaging levels, intelligently analyze packaging layouts, and accurately extract key information.

[0006] Therefore, there is an urgent need for a method that can break free from the reliance on barcodes or RFID tags, adapt to medical consumables with multiple packaging levels, and achieve intelligent identification and automatic registration of key static and dynamic information, so as to improve the automation and accuracy of medical consumables warehouse management. Summary of the Invention

[0007] To achieve the above objectives, this invention discloses a method for intelligent registration of batch entry and exit information of medical consumables, comprising the following steps: A camera array is arranged in the medical consumables storage area. Multiple images of medical consumables are acquired from multiple angles through the camera array. OCR text recognition processing is performed on the multiple medical consumables images to obtain multiple preprocessed images. One image is selected as the first image, and the remaining images are mapped onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image. The global image is input into the image segmentation model. By setting prompt words, the global image is segmented into several packaging regions with specific identifiers. OCR text extraction is performed on each packaging region to obtain the text information on the packaging surface of the packaging region. The text information on the packaging surface of each packaging area is divided into static core information and dynamic core information, and multi-dimensional matching and recognition are performed in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching. If the highest score of the cumulative score is higher than the set threshold, the physical location of the static and dynamic core information of the packaging surface text information corresponding to the highest score is checked in the global image, and the key information for medical consumables entry and exit registration is determined based on the check results. If the highest cumulative score is lower than a set threshold, the medical consumables information document is downloaded from the Medical Device Unique Identifier Database website published by the National Medical Products Administration, and a knowledge graph is constructed. Multi-channel hybrid retrieval is used on the knowledge graph to determine the key information for medical consumables entry and exit registration.

[0008] In some embodiments, during the OCR text recognition processing, the recognition result is converted according to the format requirements of a preset standard text template, and then replaced and overlaid on the original medical consumable image to form multiple pre-processed images to improve image quality.

[0009] In some embodiments, the recognition results are converted according to the format requirements of a preset standard text template, specifically as follows: In the medical consumables image after OCR recognition, the smallest outer rectangle of the text area is located and a text mask is generated. After extracting the text content and font features, it is converted into a preset sans-serif standard font, standard font size and unified typesetting format.

[0010] In some embodiments, after obtaining multiple preprocessed images, the method further includes: SuperPoint deep learning is used to extract feature points between adjacent images in multiple preprocessed images, and LightGlue model is used for feature matching to obtain matching points between images; Initial values ​​for the homography relationship matrix between images are set using matching points, and then the homography relationship is optimized according to the following formula: in, The reprojection error between matching point pairs after homography transformation is given by M, where M is the number of matching image pairs, and m is the number of matching point pairs. ij H represents the number of matching points between the i-th image and the j-th image. ij Let x be the homography transformation matrix from pixel i in the i-th image to pixel j in the j-th image. ij （ki） Let x be the k-th matching point on the i-th image and the j-th image. ij （kj） Let k be the k-th matching point between the j-th image and the i-th image; Based on the closure property of homography, the homography relationship between the first image and the remaining images is established, and the remaining images are mapped onto the first image to form a global image.

[0011] In some embodiments, the step of performing multi-dimensional matching and identification in a multi-level matching query library specifically includes: A standard database covering all dimensions of core information on medical consumables is constructed. The content of the standard database is subjected to word frequency statistical analysis. Redundant words in the medical field-specific stop word list are filtered out. The resulting first full-dimensional database is used as the coarse-grained query layer in the multi-level matching query library. The jieba Chinese word segmentation algorithm is used to split the text of the first full-dimensional database to obtain the core dimensions used to distinguish information categories. Based on the core dimensions, the first full-dimensional database is clustered and grouped. Under each cluster group, based on the similarity calculation of Chinese and English texts in Jacccard, an edit distance correction factor is introduced to adjust the similarity results by weighting, and a hierarchical similarity threshold is set for screening and judgment. The resulting second full-dimensional database is used as the fine-grained query layer in the multi-level matching query library. Coarse-grained matching is performed in the first full-dimensional database using static core information. The cluster group to which the cluster belongs is determined based on the matching results. Under the cluster group, fine-grained matching is performed in the second full-dimensional database using dynamic core information.

[0012] In some embodiments, the construction of the knowledge graph specifically includes: For the downloaded medical consumables information documents, a streaming traversal is used to extract all abbreviation tags with unique initial letters of pinyin. In accordance with the medical device unique identifier standard, a semantic mapping dictionary between the tags and their Chinese business meanings is constructed and stored. Using the unique record information of each consumable category as the head node, the abbreviation label is converted into Chinese relation using a semantic mapping dictionary. The label text is extracted as the tail node and special characters are cleaned to generate a standardized triple in the format of head node-Chinese relation-tail node.

[0013] In some embodiments, after constructing the knowledge graph, the index segmentation is optimized in the Neo4j graph database, a keyword segmenter is specified, and the consumable name, consumable specification, and registration certificate number are used as complete tokens to participate in the retrieval index. The multi-channel hybrid retrieval specifically involves: retrieving the consumable name, specifications, and registration certificate number of consumables through attribute tracing to recall consumable nodes; obtaining manufacturer information through multi-hop connections in the knowledge graph; determining the similarity between various devices using the vectorized index of medical devices; constructing aligned links for device nodes with high similarity; calculating the similarity between the search content and the recall results and sorting them in descending order; and selecting the most similar matching result as the search result.

[0014] In some embodiments, the key information for the medical consumables entry and exit registration includes at least the consumable's unique identifier, product name, manufacturer, specifications, registration certificate number, production date, expiration date, and production batch number.

[0015] This invention also discloses an intelligent registration system for bulk entry and exit information of medical consumables, comprising the following modules: The image stitching module is used to deploy a camera array in the medical consumables storage area, acquire multiple images of medical consumables from multiple angles through the camera array, perform OCR text recognition processing on the multiple medical consumables images respectively to obtain multiple preprocessed images, select one image as the first image, and map the remaining images onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image. The image segmentation module is used to input the global image into the image segmentation model, segment the global image into several packaging regions with specific identifiers by setting prompt words, and perform OCR text extraction processing on each packaging region to obtain the text information on the packaging surface of the packaging region. The information matching module is used to divide the text information on the packaging surface of each packaging area into static core information and dynamic core information, and to perform multi-dimensional matching and recognition in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching. The key information verification module is used to verify the physical location of static and dynamic core information in the global image for the packaging surface text information corresponding to the highest score if the highest score of the cumulative score is higher than a set threshold, and to determine the key information for medical consumables entry and exit registration based on the verification results. The key information supplementation module is used to download medical consumable information documents from the medical device unique identifier database website published by the National Medical Products Administration if the highest score of the cumulative score is lower than a set threshold, and to construct a knowledge graph. On the knowledge graph, a multi-channel hybrid retrieval is used to determine the key information for medical consumable entry and exit registration.

[0016] The present invention also discloses a communication terminal, including at least one processor and a memory, wherein the memory stores a computer program, and the at least one processor executes the computer program stored in the memory, thereby enabling the at least one processor to execute the above-described intelligent registration method for batch entry and exit information of medical consumables.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention uses multiple cameras to capture images of medical consumable packaging, stitches together multiple images to form a global image using image homography transformation, segments the global image using a segmentation model, performs OCR information recognition based on the segmented packaging area, and then constructs a multi-level matching query library to match key information and builds a knowledge graph to supplement the matching of new consumables. This enables batch and accurate matching of corresponding consumable information through the outer packaging when registering medical consumables' entry and exit information, thereby improving the efficiency and accuracy of entry and exit registration.

[0018] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the intelligent registration method for batch entry and exit information of medical consumables in this invention. Detailed Implementation

[0020] To make the technical means, inventive features, objectives, and effects of the invention readily understandable, the invention is further illustrated below with reference to specific figures. However, the invention is not limited to the embodiments described below.

[0021] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0022] The first embodiment of the present invention provides a method for intelligent registration of batch entry and exit information of medical consumables, such as... Figure 1 As shown, it includes the following steps: Step S1: Image acquisition, recognition, and stitching; A camera array is deployed in the medical consumables storage area to acquire multiple images of medical consumables from multiple angles. OCR text recognition processing is performed on each of the multiple medical consumables images to obtain multiple preprocessed images. One image is selected as the first image, and the remaining images are mapped onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image.

[0023] Specifically, an m*n camera array acquisition device is set up in the area covering medical consumable stacks or shelf units to simultaneously and batch acquire images of medical consumables from multiple angles. When performing OCR text recognition processing on the images, the text areas on the surface of the medical consumable packaging are easily affected by differences in lighting and inconsistent fonts, which can impact the accuracy of subsequent feature matching. Therefore, a preset standard text template is introduced to transform the recognized content. For example, the OCR text recognition results are converted according to the format requirements of the standard text template before being overlaid onto the original medical consumable image to improve image quality. The specific procedure is as follows: The smallest outer rectangle of the text area is located in the OCR-recognized medical consumable image, and a text mask is generated. After extracting the text content and font features, it is converted into a preset sans-serif standard font, standard font size, and unified typesetting format. For the text area mask, a direct replacement operation is performed. The image converted from the standard text template undergoes homography transformation to align with the text mask area of ​​the original image, covering the original text pixels. For non-text areas, the lighting and texture of the original image are preserved. This eliminates problems such as blurry text and color differences caused by variations in shooting angle and lighting intensity in different images.

[0024] After obtaining multiple preprocessed images, SuperPoint deep learning is used to extract feature points from adjacent images. The LightGlue model is then used for feature matching to obtain matching points between the images. Adjacent images are those captured by two physically adjacent cameras in a camera array. To improve the accuracy of feature point extraction, these preprocessed images undergo channel conversion and normalization: the images are converted from BGR format to RGB format, pixel values ​​are normalized to the 0-1 range, and the images are converted to grayscale. The channel-converted and normalized images are then input into SuperPoint to obtain keypoints and their corresponding descriptors. Finally, adjacent images are input into the LightGlue model to calculate matching points.

[0025] Next, the initial value of the homography relationship matrix between images is set using the matching points, and then the homography relationship is optimized according to the following formula: Where M is the number of matched image pairs, m ij H represents the number of matching points between the i-th image and the j-th image. ij H is the homography transformation matrix from pixel i in the i-th image to pixel j in the j-th image. ij (3) represents the third row of the homography transformation matrix from pixel i to pixel j. Since the homography transformation matrix is ​​a 3x3 matrix, the third row here is the last row. ij （ki） Let x be the k-th matching point on the i-th image and the j-th image. ij （kj） Let be the k-th matching point on the j-th image and the i-th image. To match the reprojection error between point pairs after homography transformation, the optimization condition is: when When the value is within the preset error threshold range, the corresponding homography transformation matrix is ​​determined to be the optimal value; wherein, the error threshold range can be adaptively adjusted according to the image resolution and scene complexity, and is generally set to 0~5 pixels to balance stitching accuracy and algorithm efficiency.

[0026] The formula for constructing the homography relationship is as follows: in, Represents the range from the i0th image to the ith image. kThe global homography transformation matrix of the first image is obtained by chaining the local homography transformation matrices between each pair of adjacent images to construct the global homography transformation matrix between non-adjacent images. Based on the closure property of homography, the homography transformation relationship between the first image and the remaining images is established, and the remaining images are all mapped onto the first image to form the global image.

[0027] Step S2, image segmentation and text extraction; The global image is input into the image segmentation model. By setting prompt words, the global image is segmented into several packaging regions with specific identifiers. OCR text extraction is performed on each packaging region to obtain the text information on the packaging surface of the packaging region.

[0028] Specifically, the global image segmentation uses the SAM3 segmentation network model, which uses prompts to define specific identifiers and segments the global image into multiple packaging regions based on these identifiers. For example, the stitched global image is input into the SAM3 model, and the model is guided by custom text prompts to identify and output the masks of all packaging boxes in the image. Then, through mask contour extraction, connected component analysis, and filtering, the global image is divided into multiple independent packaging regions, with each region corresponding to one packaging box.

[0029] The prompt words can be set according to the scenario. For example, in the scenario of general goods, "packaging box", "cardboard box", "goods box" or "box" can be used as prompt words; in the scenario of goods of specific color or size, additional qualifying words such as "blue packaging", "white box", or "standard size cardboard box" can be added to further improve the segmentation accuracy.

[0030] To achieve a reasonable division of the packaging area, other auxiliary processing is performed after segmentation. For example, noise filtering filters masks with an area smaller than a preset threshold to avoid missegmentation caused by image noise, local reflections, and other factors. Adhesion separation processing separates masks that are stuck together using methods such as contour convex hull and distance transformation to ensure that each region corresponds to only one packaging box. Region marking processing marks the image region corresponding to each mask as an independent packaging region for subsequent OCR text extraction processing.

[0031] Step S3: Multi-level matching of image and text; The text information on the packaging surface of each packaging area is divided into static core information and dynamic core information, and multi-dimensional matching and recognition are performed in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching.

[0032] In this embodiment, the static core information includes product name, manufacturer, specifications, and registration certificate number, while the dynamic core information includes production date, expiration date, and production batch number. This information is then used for matching and identification.

[0033] Before information matching, a multi-level matching query library needs to be built, including a coarse-grained query layer and a fine-grained query layer. The specific construction method of the multi-level matching query library is as follows: First, a standard database covering all dimensions of core information about medical consumables is constructed. The content in the standard database is subjected to word frequency statistical analysis to filter out redundant words in the medical field-specific stop word list. The resulting first full-dimensional database is used as the coarse-grained query layer in the multi-level matching query library. The jieba Chinese word segmentation algorithm is used to split the text of the first full-dimensional database to obtain the core dimensions used to distinguish information categories. The first full-dimensional database is then clustered and grouped based on the core dimensions.

[0034] Specifically, a standard database covering all dimensions of core information on medical consumables collects all important attributes of currently used medical consumables, forming structured records to ensure that subsequent information matching is based on complete and standardized data, avoiding matching failures due to missing information. Word frequency analysis involves calculating the frequency of each word in all text fields of the database. For example, "disposable" might appear in 90% of the records, while "syringe" might appear in 30%. Stop word filtering removes medical-specific stop words, such as "medical" and "product," which do not actually help distinguish different types of consumables. The jieba Chinese word segmentation algorithm is used to split the text into meaningful word units. From all the segmentation results, the attributes that best distinguish different consumable categories can be extracted. These attributes are the core dimensions for distinguishing information categories, such as category, specification, feature, and material dimensions. Based on these core dimensions, clustering algorithms automatically group similar consumables together, thus forming a hierarchical grouping structure where consumables within each group are very similar in core dimensions.

[0035] Secondly, under each cluster group, based on the similarity calculation of Chinese and English texts in Jacccard, an edit distance correction factor is introduced to adjust the similarity results by weighting, and a hierarchical similarity threshold is set for screening and judgment. The resulting second full-dimensional database is used as the fine-grained query layer in the multi-level matching query library.

[0036] Specifically, the system is divided into multiple groups through clustering. Consumables within each group are highly similar in core features. For example, if there is a group for 5ml sterile syringes, all products related to 5ml sterile syringes can be finely matched within this group, avoiding interference and performance overhead from global searches. Within each cluster, Jacccard similarity is used to measure the overlap between two sets, quickly assessing the overlap of characters between two texts. Edit distance measures the minimum number of single-character edits required to convert one string into another, including insertion, deletion, and replacement. In this embodiment, Jacccard similarity and edit distance-based similarity are weighted and fused. For example, the final similarity = α * Jaccard_sim + (1-α) * Edit_sim, with the weighting factor α set to 0.7. Setting a tiered similarity threshold means not using a single threshold, but setting multiple levels. For example, a similarity greater than 0.95 is considered the same product, a similarity between 0.85 and 0.95 is considered different specifications of the same product, and the same product with markings is considered. A similarity less than 0.7 is considered different products.

[0037] After the multi-level matching query library is constructed, coarse-grained matching is performed in the first full-dimensional database using static core information. The cluster group to which the database belongs is determined based on the matching results. Under the cluster group, fine-grained matching is performed in the second full-dimensional database using dynamic core information.

[0038] This embodiment employs a two-stage intelligent matching strategy to quickly and accurately locate target medical consumables within a multi-level query database. First, stable and broad information is used to rapidly pinpoint a small range—coarse-grained matching. Then, changing and specific information is used to precisely lock onto the target within this small range—fine-grained matching. Static core information refers to inherent and long-term inconvenient attributes of the consumables, typically used for classification and identification, such as consumable category, specifications, material, and registration certificate number. Dynamic core information refers to attributes of the consumables that change with batch, time, and usage scenario, used to precisely distinguish specific instances, such as production batch number, expiration date, sterilization date, current inventory, purchase price, and warehouse location.

[0039] Coarse-grained matching in the first full-dimensional database yields one or more clusters. Since the first full-dimensional database has already undergone stop word filtering and keyword indexing, the matching is very fast; for example, it matches the cluster for "5ml sterile register". Then, fine-grained matching in the second full-dimensional database yields unique or a very small number of perfectly matching consumable records, utilizing dynamic information for precise matching.

[0040] Following fine-grained matching, geometric features of the packaging area are introduced for auxiliary verification to further improve matching robustness. Consistency matching is performed between the geometric encoding vector of the normalized coordinates of the bounding box anchor points and the matching geometric vectors obtained from the database, resulting in a cumulative score for multi-dimensional matching. First, the anchor point coordinates of the packaging area bounding box are normalized and converted into a fixed-size (e.g., 128×128) geometric mask. Then, a pre-trained lightweight feature extraction model (such as MobileNetV4 or EfficientNet-LiteV4) encodes this geometric mask into a fixed-dimensional feature vector. Next, the cosine similarity of this feature vector with the pre-stored geometric feature vectors bound to consumable information in the database is calculated to obtain a geometric matching score. Finally, this geometric matching score is weighted and summed with the matching scores of the aforementioned static and dynamic information to obtain the cumulative score for multi-dimensional matching.

[0041] Step S4: Determine the key information for inbound and outbound registration; If the highest score of the cumulative score is higher than the set threshold, the physical position of the static core information and dynamic core information of the packaging surface text information corresponding to the highest score is checked in the global image. The physical position corresponds to the normalized position of the core information on the packaging surface. Based on the check result, the key information for medical consumables entry and exit registration is determined. If the highest cumulative score is lower than a set threshold, the medical consumables information document is downloaded from the Medical Device Unique Identifier Database website published by the National Medical Products Administration, and a knowledge graph is constructed. Multi-channel hybrid retrieval is used on the knowledge graph to determine the key information for medical consumables entry and exit registration.

[0042] Specifically, the static core information of the product corresponding to the three most similar images with a cumulative score higher than a set threshold is taken, and the information is checked for consistency to determine the position of the static core information and the dynamic core information. Finally, the key information for the entry and exit of medical consumables is accurately matched to ensure the efficiency and accuracy of the entry and exit process.

[0043] If the cumulative scores are all below the set threshold, it means that the medical consumable cannot be matched from the standard database. Since the consumable is a new consumable, it is necessary to use the Medical Device Unique Identifier Database website issued by the National Medical Products Administration to download the medical consumable information from the website, construct a knowledge graph, and search for key information on the entry and exit of new consumable categories based on this knowledge graph, and supplement and update the previously constructed standard database.

[0044] The specific steps for constructing a knowledge graph are: First, for the downloaded medical consumables information documents, a streaming traversal is used to extract all unique abbreviation tags with the first letter of the pinyin. In accordance with the medical device unique identifier standard, a semantic mapping dictionary between the tags and their Chinese business meanings is constructed and stored in a fixed manner to eliminate the semantic gap of the original data.

[0045] Secondly, using the unique record information of each consumable category as the head node, the abbreviation tags are converted into Chinese relations using a semantic mapping dictionary. The tag text is extracted as the tail node and special characters are cleaned. Standardized triples in the format of head node-Chinese relation-tail node are generated and tab-separated CSV files are generated to complete the core data construction of the consumable key information knowledge graph.

[0046] After constructing the knowledge graph, the index segmentation is optimized in the Neo4j graph database. A keyword segmenter is specified, and the consumable name, consumable specification, and registration certificate number are used as complete tokens in the retrieval index to avoid the retrieval four-factor error caused by the default segmenter. A dedicated consumable information retrieval index is built. Simultaneously, a vectorized index is constructed for each medical device using a pre-trained model, and stored in a vector database. For example, the pre-trained BERT model is used to construct the vectorized representation of each medical device. The input to BERT is a serialized text composed of the medical device entity, its surrounding first-order neighbor entities, and their relationships. The vector representation of the first character in the output layer is taken as the vector representation of that device. Based on this, the vectorized representation of each device can be obtained and uniformly stored into a representation matrix, with each row of the matrix corresponding to a medical device.

[0047] The multi-channel hybrid retrieval utilizes similarity to expand relevance and achieve information complementarity. It not only retrieves relevant attribute information of the located entities but also retrieves attribute information of similar entities through alignment links, where each similar entity represents an information channel. Specifically, it retrieves consumable nodes by searching for consumable names, specifications, and registration certificate numbers through attribute tracing. Manufacturer information is obtained through multi-hop connections in the knowledge graph. The similarity between various devices is determined using a vectorized index of medical devices. Device nodes with high similarity are constructed into alignment links. The similarity between the search content and the recalled results is calculated and sorted in descending order. The most similar match is selected as the final retrieval result. In attribute tracing, it is assumed that the matched entity may be a unique identifier for a medical device. In this case, the corresponding consumable name, specifications, and other attribute information can be found through the connected edges of the node. Alignment links are typically based on entity alignment, calculating the similarity of feature vectors between entities or establishing links using paths in the knowledge graph.

[0048] This embodiment acquires multi-view images through a camera array, and forms a global view through feature extraction, matching, and homography transformation optimization fusion. Based on image segmentation network and OCR, packaging text information is extracted, and a coarse-grained and fine-grained multi-level matching query library is constructed. Key information on consumable entry and exit is accurately extracted through multi-dimensional matching algorithms. For new consumable categories, a knowledge graph is constructed based on the National Medical Products Administration's Unique Device Identifier database. A multi-channel hybrid retrieval algorithm is used to obtain key information and supplement and update the private standard library, realizing efficient and accurate registration of medical consumable entry and exit information and dynamic improvement of library data.

[0049] The second embodiment of the present invention provides an intelligent registration system for batch entry and exit information of medical consumables, including an image stitching module, an image segmentation module, an information matching module, a key information verification module, and a key information supplementation module.

[0050] The image stitching module is used to deploy a camera array in the medical consumables storage area, acquire multiple images of medical consumables from multiple angles through the camera array, perform OCR text recognition processing on the multiple medical consumables images respectively to obtain multiple preprocessed images, select one image as the first image, and map the remaining images onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image. The image segmentation module is used to input the global image into the image segmentation model, segment the global image into several packaging regions with specific identifiers by setting prompt words, and perform OCR text extraction processing on each packaging region to obtain the text information on the packaging surface of the packaging region. The information matching module is used to divide the text information on the packaging surface of each packaging area into static core information and dynamic core information, and to perform multi-dimensional matching and recognition in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching. The key information verification module is used to verify the physical location of static and dynamic core information in the global image for the packaging surface text information corresponding to the highest score if the highest score of the cumulative score is higher than a set threshold, and to determine the key information for medical consumables entry and exit registration based on the verification results. The key information supplementation module is used to download medical consumable information documents from the medical device unique identifier database website published by the National Medical Products Administration if the highest score of the cumulative score is lower than a set threshold, and to construct a knowledge graph. On the knowledge graph, a multi-channel hybrid retrieval is used to determine the key information for medical consumable entry and exit registration.

[0051] Since the first embodiment corresponds to this embodiment, this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment remain valid in this embodiment, and the technical effects achievable in the first embodiment can also be achieved in this embodiment. To reduce repetition, they will not be repeated here. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0052] A third embodiment of the present invention provides a communication terminal, including at least one processor and a memory, wherein the memory stores a computer program, and the at least one processor executes the computer program stored in the memory, such that the at least one processor performs the method described in the first embodiment.

[0053] A fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the method described in the first embodiment.

[0054] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for intelligent registration of batch entry and exit information of medical consumables, characterized in that, Includes the following steps: A camera array is arranged in the medical consumables storage area. Multiple images of medical consumables are acquired from multiple angles through the camera array. OCR text recognition processing is performed on the multiple medical consumables images to obtain multiple preprocessed images. One image is selected as the first image, and the remaining images are mapped onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image. The global image is input into the image segmentation model. By setting prompt words, the global image is segmented into several packaging regions with specific identifiers. OCR text extraction is performed on each packaging region to obtain the text information on the packaging surface of the packaging region. The text information on the packaging surface of each packaging area is divided into static core information and dynamic core information, and multi-dimensional matching and recognition are performed in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching. If the highest score of the cumulative score is higher than the set threshold, the physical location of the static and dynamic core information of the packaging surface text information corresponding to the highest score is checked in the global image, and the key information for medical consumables entry and exit registration is determined based on the check results. If the highest cumulative score is lower than a set threshold, the medical consumables information document is downloaded from the Medical Device Unique Identifier Database website published by the National Medical Products Administration, and a knowledge graph is constructed. Multi-channel hybrid retrieval is used on the knowledge graph to determine the key information for medical consumables entry and exit registration.

2. The intelligent registration method for batch entry and exit information of medical consumables according to claim 1, characterized in that, In the OCR text recognition process, the recognition results are converted according to the format requirements of a preset standard text template, and then replaced and overlaid on the original medical consumable image to form multiple pre-processed images to improve image quality.

3. The intelligent registration method for batch entry and exit information of medical consumables according to claim 2, characterized in that, The recognition results are then converted according to the format requirements of a preset standard text template, specifically as follows: In the medical consumables image after OCR recognition, the smallest outer rectangle of the text area is located and a text mask is generated. After extracting the text content and font features, it is converted into a preset sans-serif standard font, standard font size and unified typesetting format.

4. The intelligent registration method for batch entry and exit information of medical consumables according to claim 1, characterized in that, After obtaining multiple preprocessed images, the method further includes: SuperPoint deep learning is used to extract feature points between adjacent images in multiple preprocessed images, and LightGlue model is used for feature matching to obtain matching points between images; Initial values ​​for the homography relationship matrix between images are set using matching points, and then the homography relationship is optimized according to the following formula: Where M is the number of matched image pairs, m ij H represents the number of matching points between the i-th image and the j-th image. ij H is the homography transformation matrix from pixel i in the i-th image to pixel j in the j-th image. ij (3) represents the third row of the homography transformation matrix from pixel i to pixel j in the image, x ij （ki） Let x be the k-th matching point on the i-th image and the j-th image. ij （kj） Let k be the k-th matching point between the j-th image and the i-th image; Based on the closure property of homography, the homography relationship between the first image and the remaining images is established, and the remaining images are mapped onto the first image to form a global image.

5. The intelligent registration method for batch entry and exit information of medical consumables according to claim 1, characterized in that, The multi-dimensional matching and identification in the multi-level matching query database specifically includes: A standard database covering all dimensions of core information on medical consumables is constructed. The content of the standard database is subjected to word frequency statistical analysis. Redundant words in the medical field-specific stop word list are filtered out. The resulting first full-dimensional database is used as the coarse-grained query layer in the multi-level matching query library. The jieba Chinese word segmentation algorithm is used to split the text of the first full-dimensional database to obtain the core dimensions used to distinguish information categories. Based on the core dimensions, the first full-dimensional database is clustered and grouped. Under each cluster group, based on the similarity calculation of Chinese and English texts in Jacccard, an edit distance correction factor is introduced to adjust the similarity results by weighting, and a hierarchical similarity threshold is set for screening and judgment. The resulting second full-dimensional database is used as the fine-grained query layer in the multi-level matching query library. Coarse-grained matching is performed in the first full-dimensional database using static core information. The cluster group to which the cluster belongs is determined based on the matching results. Under the cluster group, fine-grained matching is performed in the second full-dimensional database using dynamic core information.

6. The intelligent registration method for batch entry and exit information of medical consumables according to claim 1, characterized in that, The construction of the knowledge graph specifically involves: For the downloaded medical consumables information documents, a streaming traversal is used to extract all abbreviation tags with unique initial letters of pinyin. In accordance with the medical device unique identifier standard, a semantic mapping dictionary between the tags and their Chinese business meanings is constructed and stored. Using the unique record information of each consumable category as the head node, the abbreviation label is converted into Chinese relation using a semantic mapping dictionary. The label text is extracted as the tail node and special characters are cleaned to generate a standardized triple in the format of head node-Chinese relation-tail node.

7. The intelligent registration method for batch entry and exit information of medical consumables according to claim 6, characterized in that, After constructing the knowledge graph, the index segmentation is optimized in the Neo4j graph database. A keyword segmenter is specified, and the consumable name, consumable specification, and registration certificate number are used as complete tokens to participate in the retrieval index. The multi-channel hybrid retrieval specifically involves: retrieving the consumable name, specifications, and registration certificate number of consumables through attribute tracing to recall consumable nodes; obtaining manufacturer information through multi-hop connections in the knowledge graph; determining the similarity between various devices using the vectorized index of medical devices; constructing aligned links for device nodes with high similarity; calculating the similarity between the search content and the recall results and sorting them in descending order; and selecting the most similar matching result as the search result.

8. The intelligent registration method for batch entry and exit information of medical consumables according to claim 1, characterized in that, The key information for the registration of medical consumables entering and leaving the warehouse includes at least the consumable's unique identifier, product name, manufacturer, specifications, registration certificate number, production date, expiration date, and production batch number.

9. A smart registration system for bulk entry and exit information of medical consumables, characterized in that, Includes the following modules: The image stitching module is used to deploy a camera array in the medical consumables storage area, acquire multiple images of medical consumables from multiple angles through the camera array, perform OCR text recognition processing on the multiple medical consumables images respectively to obtain multiple preprocessed images, select one image as the first image, and map the remaining images onto the first image according to the homography transformation relationship to obtain a global image from the perspective of the first image. The image segmentation module is used to input the global image into the image segmentation model, segment the global image into several packaging regions with specific identifiers by setting prompt words, and perform OCR text extraction processing on each packaging region to obtain the text information on the packaging surface of the packaging region. The information matching module is used to divide the text information on the packaging surface of each packaging area into static core information and dynamic core information, and to perform multi-dimensional matching and recognition in a multi-level matching query library to obtain the cumulative score of multi-dimensional matching. The key information verification module is used to verify the physical location of static and dynamic core information in the global image for the packaging surface text information corresponding to the highest score if the highest score of the cumulative score is higher than a set threshold, and to determine the key information for medical consumables entry and exit registration based on the verification results. The key information supplementation module is used to download medical consumable information documents from the medical device unique identifier database website published by the National Medical Products Administration if the highest score of the cumulative score is lower than a set threshold, and to construct a knowledge graph. On the knowledge graph, a multi-channel hybrid retrieval is used to determine the key information for medical consumable entry and exit registration.

10. A communication terminal, characterized in that, It includes at least one processor and a memory, the memory storing a computer program, the at least one processor executing the computer program stored in the memory, causing the at least one processor to perform the intelligent registration method for batch entry and exit information of medical consumables as described in any one of claims 1 to 8.

Images