Consumables management method based on image recognition

The orthopedic consumables management method is constructed through image recognition technology, which solves the problem of separation of consumables from physical objects and information, and realizes efficient and accurate consumables management, which significantly improves the informatization level and reliability of management.

CN118507003BActive Publication Date: 2024-11-26JOINTOWN MEDICAL DEVICE GRP CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410959598.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-26
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

In the management of orthopedic consumables, the separation of physical consumables from information makes the process of distribution and inspection of goods cumbersome and time-consuming, and the existing solutions are difficult to effectively solve, affecting surgical efficiency and management efficiency.

Method used

Using the consumables management method based on image recognition, the initial consumables box image is preprocessed, the hole index information and initial consumables status are constructed, and the initial consumables template is constructed based on the pre-entered product information, and the consumables box image and template after use are automatically detected. Match, get the results of the consumables use.

Benefits of technology

It improves the efficiency and accuracy of consumables management, reduces manual operation time and labor intensity, ensures data accuracy and reliability, and improves management transparency and controllability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118507003B_ABST
    Figure CN118507003B_ABST
Patent Text Reader

Abstract

This invention discloses a consumables management method, device, and computer-readable storage medium based on image recognition. The method includes: preprocessing an initial consumables box image to obtain a target consumables box image; constructing hole position index information for each consumable hole in the consumables box based on the target consumables box image; detecting the initial consumables state of each consumable hole in the target consumables box image, wherein the initial consumables state includes having consumables and not having consumables; constructing an initial consumables template based on the hole position index information, the initial consumables state, and pre-entered consumables product information; and obtaining the consumables usage result based on a used consumables box image and the initial consumables template. This image recognition-based consumables management method enables efficient management of orthopedic consumables during the circulation process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical device technology, and in particular to a consumables management method, device and computer-readable storage medium based on image recognition. Background Art

[0002] Orthopedic consumables management faces unique challenges, one of the main reasons being that the physical consumables are always separated from their product information and are difficult to associate. General medical device consumables use sterile packaging and have UDI codes and product information for easy management. However, orthopedic consumables have the following characteristics: 1. In addition to joint consumables, trauma and spinal implants are usually packaged in non-sterile packaging. 2. Each operation requires not only implants, but also corresponding tools and instruments, which are non-consumables and have no identity information. 3. Orthopedic consumables management has the characteristics of two-way logistics, that is, unused consumables need to be returned to the distributor's warehouse.

[0003] These characteristics have led to a series of management problems. First, for distributors, it is necessary to check whether the consumables and their certificates and document information match when distributing goods for delivery. This process is cumbersome and repetitive. Trauma orthopedic consumables (such as screws and bone plates) are of various types, and the codes are engraved on the nuts and screw bodies. Sometimes a magnifying glass is needed to see the codes clearly and match them. The consumables for a trauma surgery usually require a screw box, a set of bone plates, and a set of surgical instruments and tools, and the distribution time usually takes more than an hour. After the operation, it is also necessary to count the unused consumables and all surgical tools, determine the specifications and batch numbers of the consumed consumables, and match the remaining screws one by one, which is extremely cumbersome. For tools, it is also necessary to confirm whether all the delivered tools have been returned. These tools are of different shapes and require professional knowledge to identify.

[0004] Secondly, inspection and disinfection are also difficult for the hospital. Since the physical objects are separated from the product information, the hospital needs to ensure that the delivered goods correspond to the documents provided by the distributor. This process usually requires the cooperation of the equipment department staff and the distributor to check the codes on the screws and documents one by one. The inspection time often exceeds half an hour, and the workload is huge, especially when facing dozens of surgeries a day.

[0005] Existing solutions attempt to solve the problem of separation of physical objects and information. Some try to manage orthopedic consumables through the "one item, one code" rule, but this method is difficult to implement because of the large number of screws, small size, and high cost. Others try to change all non-sterile packaging consumables to sterile packaging, but this increases logistics costs, and the selection and unpacking of consumables during surgery will affect surgical efficiency. Summary of the invention

[0006] The embodiment of the present application aims to achieve efficient management of orthopedic consumables during the circulation process by providing a consumables management method based on image recognition.

[0007] To achieve the above objectives, the present application provides a consumables management method based on image recognition, including:

[0008] Preprocessing the initial consumable box image to obtain a target consumable box image;

[0009] Constructing hole position index information about each consumable hole in the consumable box according to the target consumable box image;

[0010] Detecting the initial consumable state of each consumable hole in the target consumable box image, wherein the initial consumable state includes the state of having consumables and the state of having no consumables;

[0011] Constructing an initial consumables template according to the hole position index information, the initial consumables status and the pre-entered consumables product information;

[0012] The consumables usage result is obtained according to the used consumables box image and the initial consumables template.

[0013] In one embodiment, preprocessing the initial consumable box image to obtain the target consumable box image includes:

[0014] Rotate the initial consumables image to a target angle based on a preset mark in the initial consumables image;

[0015] Detecting the box surface boundary of the rotated initial consumable image;

[0016] The target consumables image is obtained by cropping and rotating the initial consumables image based on the box surface boundary.

[0017] In one embodiment, the preset identifier is an identification code having attribute information of the consumable box, and the identification code is set at a preset position of the consumable box;

[0018] Rotating the initial consumables image to a target angle based on a preset mark in the initial consumables image includes:

[0019] Detecting a current position of the identification code in the initial consumables image;

[0020] Calculating a rotation angle of the initial consumables image according to the target position and current position of the identification code;

[0021] The initial consumable material image is rotated to a target angle according to the rotation angle.

[0022] In one embodiment, constructing hole position index information about each consumable hole in the consumable box according to the target consumable box image includes:

[0023] Constructing a virtual coordinate system according to the target consumable box image;

[0024] Detecting the hole positions of all consumable holes in the target consumable box image;

[0025] The hole position index information of each consumable hole is generated according to the coordinate position of the hole position of the consumable hole in the virtual coordinate system.

[0026] In one embodiment, constructing an initial consumables template according to the hole position index information, the initial consumables state and the pre-recorded consumables product information includes:

[0027] Associating the initial consumable state with the corresponding hole position index information, and associating the hole position index information where the initial consumable state is that there are consumables with the corresponding consumable product information in the pre-entered consumable product information, to construct an initial consumable template;

[0028] The constructed initial consumables template is bound to the identity information of the consumables box.

[0029] In one embodiment, constructing an initial consumables template according to the hole position index information, the initial consumables state and the pre-recorded consumables product information further includes:

[0030] Generating a graphical operation interface on a user terminal based on the target consumable box image and the hole position index information;

[0031] Receive an operation instruction input by a user based on the graphical operation interface to modify the hole index information and the initial consumable state.

[0032] In one embodiment, the graphical operation interface includes:

[0033] An identification interface associated with the initial consumable state and hole position index information of the consumable hole based on the target consumable box image; and

[0034] Index interface for pre-entered consumable product information.

[0035] In one embodiment, obtaining the consumables usage result according to the used consumables box image and the initial consumables template includes:

[0036] Generate a current consumables template based on the used consumables box image;

[0037] Matching the current consumables template with the initial consumables template to obtain a consumables status difference;

[0038] The consumables usage result is returned according to the consumables status difference and the hole index information in the initial consumables template.

[0039] To achieve the above-mentioned objectives, an embodiment of the present application also proposes a consumable management device based on image recognition, including a memory, a processor, and an image recognition-based consumable management program stored in the memory and executable on the processor, wherein when the processor executes the image recognition-based consumable management program, it implements the image recognition-based consumable management method as described in any one of the above items.

[0040] To achieve the above objectives, an embodiment of the present application also proposes a computer-readable storage medium, on which a consumables management program based on image recognition is stored. When the consumables management program based on image recognition is executed by a processor, the consumables management method based on image recognition as described in any of the above items is implemented.

[0041] The consumables management method based on image recognition of the technical solution of the present application preprocesses the initial consumables box image to obtain the target consumables box image, then constructs the hole index information and initial consumables state of the consumable holes based on the target consumables box image, and constructs the initial consumables template in combination with the hole index information, the initial consumables state and the pre-entered consumables product information; finally, the consumables usage results can be obtained by using the consumables box image after use and the initial consumables template. In this way, the efficiency of consumables management can be greatly improved through automated image processing and data analysis, and the time and labor intensity of manual operation can be reduced. In addition, standardized image preprocessing, accurate hole position detection and status recording ensure the accuracy and reliability of the data. At the same time, the detailed consumables comparison template and usage result report are also convenient for the traceability and abnormal handling of consumables, and improve the transparency and controllability of management. That is to say, compared with the current consumables management solution, the consumables box management method of the technical solution of the present application has the advantages of high efficiency, accuracy and comprehensiveness, and significantly improves the information level and reliability of consumables management. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0043] Figure 1 This is a module structure diagram of an embodiment of a consumables management device based on image recognition according to the present invention;

[0044] Figure 2 The figure is a flow chart of an embodiment of a consumables management method based on image recognition according to the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0048] It should be noted that in the claims, any reference symbols placed between brackets shall not be constructed as limitations to the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.

[0049] like Figure 1 As shown, Figure 1 It is a structural diagram of a server 1 (also called a consumables management device based on image recognition) in a hardware operating environment involved in an embodiment of the present invention.

[0050] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", smart air conditioners, smart lights, smart power supplies with networking functions, AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablet computers, e-book readers, portable computers, etc.

[0051] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12 and a network interface 13.

[0052] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc., equipped on the server 1.

[0053] Furthermore, the memory 11 may include an internal storage unit of the server 1 as well as an external storage device. The memory 11 may be used not only to store application software and various data installed on the server 1, such as the code of the consumables management program 10 based on image recognition, but also to temporarily store data that has been output or is to be output.

[0054] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 11, such as executing a consumables management program 10 based on image recognition.

[0055] The network interface 13 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface), which is usually used to establish a communication connection between the server 1 and other electronic devices.

[0056] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth (BlueTooth) communication protocol or a combination thereof.

[0057] Optionally, the server may further include a user interface, which may include a display, an input unit such as a keyboard, and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be referred to as a display screen or a display unit, which is used to display information processed in the server 1 and to display a visual user interface.

[0058] Figure 1 Only the server 1 having components 11-13 and the consumables management program 10 based on image recognition is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0059] In this embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0060] Preprocessing the initial consumable box image to obtain a target consumable box image;

[0061] Constructing hole position index information about each consumable hole in the consumable box according to the target consumable box image;

[0062] Detecting the initial consumable state of each consumable hole in the target consumable box image, wherein the initial consumable state includes the state of having consumables and the state of having no consumables;

[0063] Constructing an initial consumables template according to the hole position index information, the initial consumables status and the pre-entered consumables product information;

[0064] The consumables usage result is obtained according to the used consumables box image and the initial consumables template.

[0065] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0066] Rotate the initial consumables image to a target angle based on a preset mark in the initial consumables image;

[0067] Detecting the box surface boundary of the rotated initial consumable image;

[0068] The target consumables image is obtained by cropping and rotating the initial consumables image based on the box surface boundary.

[0069] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0070] Detecting a current position of the identification code in the initial consumables image;

[0071] Calculating a rotation angle of the initial consumables image according to the target position and current position of the identification code;

[0072] The initial consumable material image is rotated to a target angle according to the rotation angle.

[0073] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0074] Constructing a virtual coordinate system according to the target consumable box image;

[0075] Detecting the hole positions of all consumable holes in the target consumable box image;

[0076] The hole position index information of each consumable hole is generated according to the coordinate position of the hole position of the consumable hole in the virtual coordinate system.

[0077] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0078] Associating the initial consumable state with the corresponding hole position index information, and associating the hole position index information where the initial consumable state is that there are consumables with the corresponding consumable product information in the pre-entered consumable product information, to construct an initial consumable template;

[0079] The constructed initial consumables template is bound to the identity information of the consumables box.

[0080] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0081] Generating a graphical operation interface on a user terminal based on the target consumable box image and the hole position index information;

[0082] Receive an operation instruction input by a user based on the graphical operation interface to modify the hole index information and the initial consumable state.

[0083] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0084] An identification interface associated with the initial consumable state and hole position index information of the consumable hole based on the target consumable box image; and

[0085] Index interface for pre-entered consumable product information.

[0086] In one embodiment, the processor 12 may be used to call the consumables management program based on image recognition stored in the memory 11, and perform the following operations:

[0087] Generate a current consumables template based on the used consumables box image;

[0088] Matching the current consumables template with the initial consumables template to obtain a consumables status difference;

[0089] The consumables usage result is returned according to the consumables status difference and the hole index information in the initial consumables template.

[0090] Based on the hardware architecture of the above-mentioned consumables management device based on image recognition, an embodiment of the consumables management method based on image recognition of the present invention is proposed. The consumables management method based on image recognition of the present invention aims to achieve efficient management of orthopedic consumables.

[0091] Reference Figure 2 , Figure 2 This is an embodiment of the consumables management method based on image recognition of the present invention, and the consumables management method based on image recognition includes the following steps:

[0092] S10, preprocessing the initial consumable box image to obtain a target consumable box image.

[0093] Among them, consumable boxes refer to boxes or containers used to store and manage medical consumables such as syringes, surgical tools, reagents, etc. These consumable boxes are designed in various shapes and sizes to facilitate organization and management in medical work.

[0094] The initial consumable box refers to the state where all the expected consumables are contained in the consumable box before it is used. In this state, each hole in the consumable box is filled with corresponding consumables according to the requirements (this requirement can be the manufacturer's consumable box design standard or a custom standard based on the personalized needs of the hospital). The types, quantities and positions of these consumables are fixed at the factory or at the time of initialization.

[0095] Specifically, the image of the initial consumable box can be captured by a high-resolution camera to obtain the image of the initial consumable box. For example, the image of the initial consumable box can be obtained by capturing a top view of the initial consumable box.

[0096] Furthermore, the initial consumable box image may be pre-processed by rotation, cropping, graying, filtering, super-resolution, etc. to obtain a target consumable box image.

[0097] It is understandable that by preprocessing the initial consumable box image, a consumable box image in a standardized format can be obtained, thereby ensuring that the consumable box image is highly accurate and consistent, which in turn facilitates subsequent identification, processing and comparison to improve the accuracy of consumable management.

[0098] S20: Construct hole position index information about each consumable hole in the consumable box according to the target consumable box image.

[0099] Among them, consumable holes refer to the small holes or positions in the consumable box for placing consumables. These holes are part of the consumable box design and are specifically used to fix and store various consumables (such as screw consumables, syringes, etc.) to ensure that the consumables will not move or get lost during transportation and use.

[0100] The hole index information is a detailed description and record of each consumable hole in the consumable box, including its location, status, and associated consumable information. This information is used to manage and track the usage of each consumable hole. Each hole has unique index information to distinguish different hole positions.

[0101] Specifically, the hole index information can be constructed by the following steps:

[0102] 1. Shape detection: Use Hough Transform to detect circular or elliptical holes in the target consumables image. Hough Transform can effectively identify circular or elliptical objects in the image.

[0103] 2. Coordinate system construction:

[0104] 2.1. Determine the reference point: Select a fixed point in the image (such as the QR code position) as the origin of the coordinate system.

[0105] 2.2 Coordinate calculation: Calculate the two-dimensional coordinates (X, Y) of each hole position in the image based on the reference point and the scale of the image.

[0106] 2.3. Coordinate system conversion: Convert the image coordinates into the physical coordinates (such as millimeters, centimeters, etc.) in the actual consumables box.

[0107] 3. Index information generation:

[0108] 3.1. Hole position numbering: Assign a unique number to each detected hole position.

[0109] 3.2. Information recording: record the number, two-dimensional coordinate position and status (with or without consumables) of each hole position.

[0110] 3.3. Index generation: Generate an index file (such as CSV, JSON, etc.) containing all hole position information to facilitate subsequent management and query.

[0111] S30, detecting an initial consumable state of each consumable hole in the target consumable box image, wherein the initial consumable state includes a state with consumables and a state without consumables.

[0112] The initial consumables status refers to the status of the consumables in each consumable hole before the consumable box is used. This is the reference point for consumables management and is used for comparison and tracking during subsequent use. The initial consumables status mainly includes the following two situations:

[0113] There are consumables: It means that a specified medical consumable is placed in a consumable hole, such as screw consumables, syringes, etc.

[0114] No filament: Indicates that a filament hole is empty and no filament is placed.

[0115] Specifically, image analysis algorithms (such as color analysis and shape matching) may be used to detect whether consumables are present in each well.

[0116] S40, constructing an initial consumables template according to the hole position index information, the initial consumables status and the pre-recorded consumables product information.

[0117] Among them, pre-entering consumable product information means entering all possible consumable product information into the system in advance. For example, consumable product information includes but is not limited to consumable name, specification, model, product identification (i.e. UDI code, UDI code is the unique identification of medical devices promoted by the Food and Drug Administration), certificate image, etc.

[0118] Optionally, the consumable product information can be entered into the terminal by manual input, code scanning, batch import, RFID (radio frequency identification) entry, etc.

[0119] Furthermore, the initial consumables template is a reference file or data structure that records and maps the status of each well in the consumable box and the corresponding consumables information in detail. It is used to track and manage the storage and use of consumables, ensuring that the consumable status and information of each well are clearly visible.

[0120] Specifically, you can build an initial consumables template by following the steps below:

[0121] 1. Data collection and integration:

[0122] 1.1. Hole index information: The position and identification information of each hole in the consumable box is obtained through the previous steps (such as image recognition and coordinate calculation).

[0123] 1.2. Initial consumable status: Check the initial status of each hole (with consumables or without consumables) and record it.

[0124] 1.3. Pre-enter consumable product information: Pre-enter all possible consumable product information into the system, including name, specification, batch, production date, etc.

[0125] 2. Database Management System (DBMS):

[0126] 2.1. Choose DBMS: Choose a suitable database management system, such as MySQL, PostgreSQL, MongoDB, etc., to store and manage all relevant information.

[0127] 2.2. Create database and table: Create a database and corresponding table structure in DBMS to store hole index information, initial consumable status and consumable product information.

[0128] 3. Data processing and matching:

[0129] 3.1. Programming language: Use Python, Java and other programming languages ​​to write scripts to implement data processing and matching logic.

[0130] 3.2. Data matching: According to the initial state in the hole index information, the hole positions with consumables are matched with the pre-entered consumable product information, and the matching results are stored.

[0131] 4. Template generation:

[0132] 4.1. Generate template: Generate the initial consumables template based on the matching results. It can be a file in JSON, XML or other formats for easy storage and management.

[0133] 4.2. Template storage: Store the generated initial consumable template in the system to ensure quick retrieval and application for subsequent use.

[0134] In this way, an initial consumable template is constructed by using a database management system (DBMS) and a programming language.

[0135] S50: Obtain consumables usage results according to the used consumables box image and the initial consumables template.

[0136] Specifically, the status of the consumables in the current consumable box can be determined based on the image of the used consumable box, and then the status of the consumables in the current consumable box can be compared with the status of the consumables in the initial consumable box by combining the hole index information and the consumable product information to determine the consumable usage status in each consumable hole and obtain the exact consumable usage results.

[0137] Specifically, in some embodiments, obtaining the consumables usage result according to the image of the used consumables box and the initial consumables template includes:

[0138] S51 . Generate a current consumables template according to the used consumables box image.

[0139] Specifically, reference may be made to the embodiments of step S10 to step S40 to generate a current consumables template according to the used consumables box image.

[0140] S52: Match the current consumables template with the initial consumables template to obtain a consumables status difference.

[0141] Specifically, an initial consumable template in an initial state may be read from a database or a storage file.

[0142] Then, each well position in the current consumable template is positionally matched with the corresponding well position in the initial consumable reference template. The matching method can be based on a predefined coordinate system to ensure that each well position accurately corresponds in the two templates.

[0143] Next, the consumable status of each matching hole in the current template and the initial template is compared to identify the change in status. For example, it is detected that the status of a hole changes from "with consumable" to "without consumable" or from "without consumable" to "with consumable".

[0144] Finally, all detected differences in the consumable status are recorded to form a difference list. Each difference item contains the location, initial state and current state of the hole.

[0145] S53: Return the consumables usage result according to the consumables status difference and the hole position index information in the initial consumables template.

[0146] Specifically, after determining the difference in the status of the consumables in the consumable box, the recorded differences in the status of the consumables are analyzed to determine the specific situation of each difference. For example, if the status of the consumables of a certain hole position changes from "consumables present" to "no consumables", it means that the consumables have been used. If the status of the consumables of a certain hole position changes from "no consumables" to "consumables present", it means that an abnormality has occurred in the current hole position.

[0147] Next, the specific location and number of each hole is described in detail according to the hole index information in the initial consumables reference template. Based on the parsed state differences and hole index information, a consumables usage report is generated. The report content includes the initial state, current state, changes, and specific location and number of each hole.

[0148] Finally, the generated consumables usage report is output to an easy-to-read format (such as a text file, PDF report, or database record) and stored in the system for medical staff or manufacturers to view and manage.

[0149] It can be understood that the consumables management method based on image recognition of the technical solution of the present application preprocesses the initial consumables box image to obtain the target consumables box image, and then constructs the hole index information and initial consumables state of the consumable holes based on the target consumables box image, and constructs the initial consumables template in combination with the hole index information, the initial consumables state and the pre-entered consumables product information; finally, the consumables use results can be obtained by using the consumables box image after use and the initial consumables template. In this way, the efficiency of consumables management can be greatly improved through automated image processing and data analysis, and the time and labor intensity of manual operation can be reduced. In addition, standardized image preprocessing, accurate hole position detection and status recording ensure the accuracy and reliability of the data. At the same time, the detailed consumables comparison template and usage result report also facilitate the traceability and abnormal handling of consumables, and improve the transparency and controllability of management. That is to say, compared with the current consumables management solution, the consumables box management method of the technical solution of the present application has the advantages of high efficiency, accuracy, and comprehensiveness, which significantly improves the information level and reliability of consumables management.

[0150] In some embodiments, preprocessing the initial consumable box image to obtain the target consumable box image includes:

[0151] S11, rotating the initial consumables image to a target angle based on a preset mark in the initial consumables image.

[0152] Among them, the preset logo is an identifier used as a reference point or benchmark in the image processing and recognition process, which can be a QR code, a barcode, a specific shape (such as dots, rectangular blocks, five-pointed stars, etc.), etc.

[0153] Specifically, the preset mark in the image can be detected by image processing technology, and then based on the position of the preset mark in the initial consumable image, a rotation transformation is applied to rotate the image to a target angle so that the preset mark is located at a predetermined position to ensure the standardized direction of the image.

[0154] S12, detecting the box surface boundary of the rotated initial consumables image.

[0155] The box surface boundary refers to the outer edge or boundary line of the surface area of ​​the consumable box. It is a key feature to determine the effective area of ​​the consumable box and distinguish it from the background or other irrelevant parts during image processing.

[0156] Specifically, edge detection algorithms (such as Canny edge detection) can be used to extract edge information in the image to highlight the border of the box surface. Then, based on the edge detection results, contour detection algorithms (such as contour tracking algorithms) are used to identify the contour of the box surface in the image. Finally, based on the detected contour, the boundary position of the box surface is confirmed to remove noise and enhance the boundary.

[0157] S13, obtaining the target consumables image by cropping and rotating the initial consumables image based on the box surface boundary.

[0158] Specifically, according to the box surface boundary detected in step S12, the parameters of the area to be cropped (such as the coordinates of the cropped rectangle) are calculated. Then, the cropping operation is applied to remove the part outside the box surface boundary in the image, and only the box surface area is retained. Finally, the cropped image is output as the target consumables image to ensure that only the valid part of the consumables box is included in the image.

[0159] It can be understood that based on the above embodiments, the consistency of the initial consumables image in direction, size and content is ensured through rotation and cropping, eliminating the differences caused by shooting angles and environmental changes. At the same time, standardization and boundary detection can significantly improve the accuracy of image recognition, reduce errors, and ensure the reliability of subsequent steps (such as hole position detection and state recognition).

[0160] Specifically, the preset identification in the present technical solution refers to an identification code with attribute information of the consumable box, and these identification codes are set at a predetermined position of the consumable box.

[0161] The attribute information of the consumable box includes but is not limited to the name, specification, batch number, manufacturing date, expiration date, manufacturer, etc. This information facilitates the identification and management of the details of the consumable box.

[0162] The identification code includes but is not limited to a QR code, a barcode, an RFID tag, etc. The identification code is fixed at a specific position on the consumable box to ensure that it can be found in the image every time it is scanned or photographed. For example, the QR code may be fixed at the upper right corner of the consumable box.

[0163] In some embodiments, rotating the initial consumable image to a target angle based on a preset mark in the initial consumable image includes:

[0164] S111, detecting the current position of the identification code in the initial consumables image.

[0165] Specifically, image recognition technology (such as a QR code or barcode recognition algorithm) can be used to locate the position of the identification code in the image. Commonly used algorithms include QR code detection and barcode detection in the OpenCV library.

[0166] Then, the coordinate information of the identification code, that is, the specific position of the identification code in the image (such as the center point coordinates), is extracted to determine the current position of the identification code.

[0167] S112: Calculate the rotation angle of the initial consumables image according to the target position and current position of the identification code.

[0168] Specifically, before preprocessing the initial consumable box image, the target position of the identification code in the standardized image may be predefined, for example, the identification code may be positioned in a specific quadrant of the image (eg, the upper left corner, ie, the fourth quadrant).

[0169] Then, use geometric calculation methods, such as trigonometric functions (atan2) to calculate the rotation angle required to get from the current position to the target position.

[0170] S113, rotating the initial consumable material image to a target angle according to the rotation angle.

[0171] Specifically, the initial consumables image is rotated to the calculated target angle by using image transformation technology (such as affine transformation) to generate and output a rotated image as a target consumables image for subsequent processing steps.

[0172] It can be understood that based on the above embodiments, the automation of image preprocessing can be achieved by automatically detecting the position of the identification code and calculating the rotation angle, which greatly improves the processing efficiency and reduces manual intervention.

[0173] At the same time, the identification code can be dynamically generated as needed, containing different information fields, which is convenient for the expansion and update of consumables information. In addition, modern image recognition technology has a very high recognition rate and recognition speed for both QR codes and barcodes, and can accurately recognize them even when the image quality is poor or partially damaged. In particular, QR codes have strong error correction capabilities and can still be correctly decoded even when they are partially blocked or damaged, improving the reliability of recognition.

[0174] Therefore, using the identification code as the preset identification has the advantages of rich information, accurate identification, efficient processing, etc. Of course, the design of the present application is not limited thereto, and in other embodiments, other identifications may also be selected as the preset identifications.

[0175] In some embodiments, constructing hole position index information about each consumable hole in the consumable box according to the target consumable box image includes:

[0176] S21. Construct a virtual coordinate system according to the target consumable box image.

[0177] Specifically, a virtual coordinate system can be constructed based on the following steps:

[0178] 1. Select a fixed reference point as the origin of the coordinate system. For example, you can select the center point of the identification code as the origin of the coordinate system.

[0179] 2. Establish the coordinate axis and determine the direction of the X-axis and Y-axis. Usually the X-axis is horizontal to the right and the Y-axis is vertical downward or upward (in the image coordinate system).

[0180] 3. Calculate the proportional relationship between the actual physical size and the image coordinates to ensure that the coordinate system can reflect the actual position.

[0181] 4. Mark out a coordinate grid in the image so that each point has a clear coordinate (X, Y).

[0182] S22, detecting the hole positions of all consumable holes in the target consumable box image.

[0183] Specifically, a shape detection algorithm (such as Hough circle transform) can be used to identify circular holes in the image, and then the center point coordinates and radius of each hole are determined, and the position of the hole is recorded.

[0184] S23. Generate hole position index information of each consumable hole according to the coordinate position of the hole position of the consumable hole in the virtual coordinate system.

[0185] Specifically, the hole index information can be generated based on the following steps:

[0186] 1. Convert coordinates: Convert the detected hole center coordinates into coordinate positions in the virtual coordinate system. Use the coordinate system established in step S21 to convert the image coordinates into actual physical coordinates.

[0187] 2. Generate index information: Record the coordinate position (X, Y) and other related information (such as hole radius, status, etc.) of each hole position to generate index information, where each hole position has a unique number.

[0188] 3. Store index information: Store the index information of all hole positions in a database or file for easy subsequent access and management.

[0189] Through the embodiments provided by the above steps S21 to S23, the hole position index information of the consumable holes can be generated based on the target consumable box image, and the precise positioning of the consumable holes can be ensured.

[0190] In some embodiments, constructing an initial consumable template according to the hole index information, the initial consumable state and the pre-recorded consumable product information includes:

[0191] S31, associating the initial consumable state with the corresponding hole position index information, and associating the hole position index information where the initial consumable state is that there are consumables with the corresponding consumable product information in the pre-entered consumable product information to construct an initial consumable template.

[0192] Specifically, the initial consumables template may be represented based on a pre-set data structure, such as a list structure or a dictionary, wherein each key (hole position index) is mapped to a corresponding value (initial consumables state and consumables information).

[0193] Then, the hole index information (i.e., the coordinates of the hole) is associated with the initial consumable status of the hole. If the initial consumable status is "consumables are available", the corresponding pre-entered consumable product information is retrieved and bound to the hole index information.

[0194] Finally, the associated initial consumable status and linked consumable information are merged into a single data structure that represents the initial consumable template for the consumable box.

[0195] S32, binding the constructed initial consumables template to the identity information of the consumables box.

[0196] The identity information of the consumable box refers to the unique identifier (ie, ID) of the consumable box. Each consumable box has a unique identifier, which can be automatically generated according to the attribute information of the consumable box, the attribute information of the consumables, etc. It is worth adding that the unique identifier can be recorded in the identifier (ie, QR code).

[0197] Specifically, by associating the initial consumable template constructed in step S31 with the ID of the corresponding consumable box, the binding of the initial consumable template with the identity information of the consumable box can be achieved.

[0198] By associating the initial consumables template with the unique identifier of the corresponding consumable box, it is possible to ensure that the initial status information is correctly associated with the specific box to facilitate subsequent consumables retrieval and management.

[0199] In some embodiments, constructing an initial consumable template according to the hole index information, the initial consumable state and the pre-recorded consumable product information further includes:

[0200] S33: Generate a graphical operation interface on a user terminal based on the target consumable box image and the hole position index information.

[0201] Specifically, after obtaining the target consumable box image and hole index information, the target consumable box image can be integrated into the GUI (Graphical User Interface) framework and layout based on the pre-built GUI, and the hole index information and its corresponding coordinates can be displayed on the target consumable box image in the form of a table or icon. The GUI framework can be Qt, Tkinter or JavaFX, etc. For GUI interaction, it can include using mouse or keyboard events to zoom in / out the image, select a specific hole, or modify the hole information.

[0202] Optionally, the graphical operation interface includes two components: an identification interface and an index interface.

[0203] The identification interface is used for visual representation of the target consumable box image, which allows the user to directly interact with the image and its corresponding elements. Specifically, each hole on the image is associated with its initial consumable state, indicated by color-coded markers:

[0204] For holes with consumables, a red frame can be used to indicate that the hole actually has consumables and needs to be associated with product information. When the hole is associated with the consumable product information, it will be changed to a green frame.

[0205] For holes that are identified as having no consumables, a white frame can be used to indicate that the hole has no consumables temporarily.

[0206] The index interface is used to display the index of pre-entered consumable product information. This list includes but is not limited to detailed information such as product type, quantity, and color. Users can use the index interface to select a product in the index and associate it with the hole clicked on the image. When this operation updates the color mark of the hole to green, it means that the association is successful.

[0207] Exemplarily, the following is an optional consumable product information association operation: click on the color box coordinates of the unassociated product information in the graphical operation interface, and select the corresponding consumable product information in the index interface. The product information can be input in a variety of ways such as input, barcode scanning and parsing, and then the consumable product information is associated with the coordinates, and the coordinate product information association is completed.

[0208] As can be understood, the color-coded marking of the hole position status provides users with immediate visual feedback, indicating whether the hole position has associated product information. The product information association mechanism enables users to seamlessly link pre-recorded product information to a specific hole position, ensuring accurate representation of the initial state of the consumable box.

[0209] S34, receiving an operation instruction input by the user based on the graphical operation interface to modify the hole index information and the initial consumable status.

[0210] Specifically, in the graphical operation interface, the user can input operation instructions through input methods such as mouse, keyboard, touch screen, stylus, voice, etc. to modify the corresponding hole index information in the data structure.

[0211] For example, the user can modify the hole index information and the initial consumable status as follows based on the graphical operation interface:

[0212] 1. Modification operation: For the green frame hole (identified as having consumables), if the identification is wrong (there is no consumables at the hole, but it is identified as having consumables due to interference factors), you can cancel the identification and change the green filled frame to a white frame without filling;

[0213] 2. Deletion operation: For the white frame hole position (identified as a hole position, no consumables), if the recognition error is made here (this is not a hole position for inserting consumables, but it is identified as a hole position due to interference factors), it can be deleted, and this coordinate will be deleted;

[0214] 3. Hole adding operation: If a certain location is a hole but is not recognized due to interference factors, you can add the hole coordinates with a white frame;

[0215] 4. Add consumables: If there is a consumable hole somewhere but the coordinates are not directly identified due to interference factors, you can directly add the green color frame hole coordinates;

[0216] 5. Merge operation: For consumables with non-unit hole positions (one consumable hole position is one unit), that is, consumables placed in the consumable board layer that cover several holes, you can merge the holes after fixing the position, that is, merge the coordinates of several units into one color frame coordinate.

[0217] It can be understood that through this graphical operation interface, a clear and intuitive visual representation of the consumable box is provided, allowing users to easily identify and interact with specific hole positions. Users can directly click on the hole positions in the image to select and conveniently modify relevant information, thereby effectively reviewing, modifying and managing the initial state of the consumable box.

[0218] In addition, the embodiment of the present invention further proposes a computer-readable storage medium, which may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a consumables management program 10 based on image recognition. The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the consumables management method based on image recognition and the server 1, and will not be repeated here.

[0219] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0220] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0222] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0223] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0224] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A consumables management method based on image recognition, It is characterized in that include: Preprocessing the initial consumable box image to obtain a target consumable box image; Constructing hole position index information about each consumable hole in the consumable box according to the target consumable box image; Detecting the initial consumable state of each consumable hole in the target consumable box image, wherein the initial consumable state includes the state of having consumables and the state of having no consumables; Constructing an initial consumables template according to the hole position index information, the initial consumables status and the pre-entered consumables product information; According to the used consumable box image and the initial consumable template, the consumable usage result is obtained; wherein, According to the target consumable box image, hole position index information about each consumable hole in the consumable box is constructed, including: Constructing a virtual coordinate system according to the target consumable box image; Detecting the hole positions of all consumable holes in the target consumable box image; Generate hole position index information of each consumable hole according to the coordinate position of the hole position of the consumable hole in the virtual coordinate system; Constructing an initial consumables template according to the hole index information, the initial consumables state and the pre-recorded consumables product information, including: Associating the initial consumable state with the corresponding hole position index information, and associating the hole position index information where the initial consumable state is that there are consumables with the corresponding consumable product information in the pre-entered consumable product information, to construct an initial consumable template; The constructed initial consumables template is bound to the identity information of the consumables box.

2. The consumables management method based on image recognition according to claim 1, It is characterized in that The initial consumable box image is preprocessed to obtain a target consumable box image, including: Rotating the initial consumable box image to a target angle based on a preset mark in the initial consumable box image; Detecting the box surface boundary of the rotated initial consumable box image; The target consumable box image is obtained by cropping and rotating the initial consumable box image based on the box surface boundary.

3. The consumables management method based on image recognition as claimed in claim 2, It is characterized in that The preset identifier is an identification code having attribute information of the consumable box, and the identification code is set at a preset position of the consumable box; Rotating the initial consumable box image to a target angle based on a preset mark in the initial consumable box image includes: Detecting a current position of the identification code in the initial consumable box image; Calculating a rotation angle of the initial consumable box image according to a target position and a current position of the identification code; The initial consumable box image is rotated to a target angle according to the rotation angle.

4. The consumables management method based on image recognition according to claim 1, It is characterized in that Constructing an initial consumables template according to the hole index information, the initial consumables state and the pre-recorded consumables product information, further comprising: Generating a graphical operation interface on a user terminal based on the target consumable box image and the hole position index information; Receive an operation instruction input by a user based on the graphical operation interface to modify the hole index information and the initial consumable state.

5. The consumables management method based on image recognition as claimed in claim 4, It is characterized in that The graphical operation interface includes: An identification interface associated with the initial consumable state and hole position index information of the consumable hole based on the target consumable box image; and Index interface for pre-entered consumable product information.

6. The consumables management method based on image recognition as claimed in claim 5, It is characterized in that According to the used consumable box image and the initial consumable template, the consumable usage result is obtained, including: Generate a current consumables template based on the used consumables box image; Matching the current consumables template with the initial consumables template to obtain a consumables status difference; The consumables usage result is returned according to the consumables status difference and the hole index information in the initial consumables template.

7. A consumables management device based on image recognition, It is characterized in that It includes a memory, a processor, and a consumables management program based on image recognition stored in the memory and executable on the processor. When the processor executes the consumables management program based on image recognition, the consumables management method based on image recognition as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a consumables management program based on image recognition, and when the consumables management program based on image recognition is executed by a processor, the consumables management method based on image recognition as described in any one of claims 1 to 6 is implemented.