Method and device for extracting index measurement value of medical instrument

Abstract

The present invention discloses a method and device for extracting the index measurement value of a medical instrument, and relates to the technical field of detection and alignment of medical instrument images. The method obtains all target bounding boxes and target texts on the medical instrument image through OCR recognition technology, and then obtains the direction of the index measurement value bounding box based on "multi-direction + multi-calculation coordinate point" screening, obtains the distance between the index measurement value bounding box and the index name bounding box according to the calculation coordinate point, and finally associates the index name and the index measurement value according to the image correspondence. The method and device of the present invention can efficiently, conveniently and accurately extract the index name, index measurement value and the corresponding relationship between the two on the medical instrument.

Description

Technical Field

[0001] The present invention relates to the technical field of medical instrument image detection and alignment, and in particular to a method and device for extracting index measurement values ​​of a medical instrument. Background Art

[0002] With the development of technology, there are more and more monitoring indicators for medical monitoring instruments on the market, from body temperature and blood pressure at the beginning to heartbeat, pulse, blood lipids, blood oxygen, blood sugar, etc. As the technical threshold of health indicator monitoring instruments is lowered, there are more and more manufacturers. To understand the health status of patients, the more comprehensive the health indicator data is, the more conducive it is to quickly identify the cause of the patient's illness and provide a diagnosis and treatment plan in an emergency. Therefore, how to quickly and accurately collect indicator data of health monitoring instruments from different manufacturers and different versions at home has become a technical problem that needs to be solved.

[0003] There are currently two ways to solve this problem on the market:

[0004] 1. Through manual recording or through app form input. The first method is relatively primitive, the data is prone to errors, and the original data is not retained. The accuracy of the data depends entirely on the hand-copying personnel, and it is difficult to trace the data. The second method has the same problem as hand-copying numbers if the original image is not saved. The second method also requires relevant training for the input personnel, such as: which value needs to be entered into which field, and the whole process is time-consuming and troublesome.

[0005] 2. By integrating the interface standards of different models of instruments from different manufacturers. Generally, manufacturers provide monitoring indicator acquisition functions based on Bluetooth, infrared, and interface services. However, this method requires studying the protocols and interface specifications of each manufacturer, which is time-consuming and may require the cooperation of each manufacturer. It is labor-intensive and may not necessarily obtain the support of all manufacturers, nor may it be possible to collect data from all instruments.

[0006] Both of the above methods cannot efficiently, conveniently and accurately extract the indicator name, indicator measurement value and the corresponding relationship between the two on the medical instrument. Summary of the invention

[0007] In view of the above-mentioned defects or deficiencies in the prior art, the present invention provides a method and device for extracting indicator measurement values ​​of a medical instrument, which can fully or partially solve the technical problems mentioned in the background technology.

[0008] One aspect of the present invention provides a method for extracting an indicator measurement value of a medical instrument, comprising:

[0009] Acquire medical instrument images with index measurements;

[0010] Performing OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image;

[0011] Determine a keyword that can represent the name of the indicator to be extracted from the target text, and determine a keyword bounding box from the target bounding box;

[0012] Taking the keyword bounding box as the central area, dividing the peripheral area surrounding the central area into multiple directions, and determining the direction of the remaining target bounding boxes relative to the central area, the direction belonging to one of the multiple directions;

[0013] A plurality of coordinate points are respectively set on the remaining target boundary boxes and the keyword boundary box, and a first coordinate point on the keyword boundary box and a second coordinate point on the remaining target boundary box are determined from the plurality of coordinate points according to the direction of the remaining target boundary box relative to the keyword boundary box; wherein the distance between the first coordinate point on the keyword boundary box and the second coordinate point on the remaining target boundary box is the minimum value of the distances between the plurality of coordinate points on the keyword boundary box and the plurality of coordinate points on the remaining target boundary boxes;

[0014] sorting the distances between the first coordinate point on the keyword bounding box and the second coordinate point on each of the remaining target bounding boxes, and taking the target bounding box with the shortest distance and the target text meeting the preset conditions as the area to be extracted that matches the keyword bounding box;

[0015] Keywords in the keyword boundary box are extracted as indicator names, and target text in the to-be-extracted area is extracted as indicator measurement values.

[0016] Further, the multiple directions include upper left, upper, upper right, right, lower right, lower, lower left and left.

[0017] Further, the multiple coordinate points include an upper left corner point, an upper middle point, an upper right corner point, a right middle point, a lower right corner point, a lower middle point, a lower left corner point and a left middle point on the target bounding box.

[0018] Furthermore, the target bounding box information includes coordinate values ​​of four corner points of the target bounding box.

[0019] Further, the step of determining a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding boxes from the plurality of coordinate points according to the directions of the remaining target bounding boxes relative to the keyword bounding box includes:

[0020] When the remaining target bounding boxes are located above the keyword bounding box, the upper midpoint of the keyword bounding box is used as the first coordinate point, and the lower midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0021] When the remaining target bounding boxes are located below the keyword bounding box, the lower midpoint of the keyword bounding box is used as the first coordinate point, and the upper midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0022] When the remaining target bounding boxes are located to the left of the keyword bounding box, the left midpoint of the keyword bounding box is used as the first coordinate point, and the right midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0023] When the remaining target bounding boxes are located to the right of the keyword bounding box, the right midpoint of the keyword bounding box is used as the first coordinate point, and the left midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0024] When the remaining target bounding boxes are located at the upper left of the keyword bounding box, the upper left corner point of the keyword bounding box is used as the first coordinate point, and the lower right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0025] When the remaining target bounding boxes are located at the upper right corner of the keyword bounding box, the upper right corner point of the keyword bounding box is used as the first coordinate point, and the lower left corner point of the remaining target bounding boxes is used as the second coordinate point;

[0026] When the remaining target bounding boxes are located at the lower left of the keyword bounding box, the lower left corner point of the keyword bounding box is used as the first coordinate point, and the upper right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0027] When the remaining target bounding boxes are located at the lower right corner of the keyword bounding box, the lower right corner point of the keyword bounding box is used as the first coordinate point, and the upper left corner point of the remaining target bounding boxes is used as the second coordinate point.

[0028] Another aspect of the present invention further provides a device for extracting index measurement values ​​of a medical instrument, comprising:

[0029] An image acquisition module configured to acquire a medical instrument image with an indicator measurement value;

[0030] A recognition module is configured to perform OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image;

[0031] A determination module is configured to determine a keyword that can represent the name of the indicator to be extracted from the target text, and determine a keyword bounding box from the target bounding box;

[0032] a direction determination module configured to take the keyword bounding box as a central area, divide the peripheral area surrounding the central area into a plurality of directions, and determine a direction of the remaining target bounding boxes relative to the central area, the direction belonging to one of the plurality of directions;

[0033] A coordinate point determination module is configured to set a plurality of coordinate points on the remaining target boundary boxes and the keyword boundary box respectively, and determine a first coordinate point on the keyword boundary box and a second coordinate point on the remaining target boundary box from the plurality of coordinate points according to the direction of the remaining target boundary box relative to the keyword boundary box; wherein the distance between the first coordinate point on the keyword boundary box and the second coordinate point on the remaining target boundary box is the minimum value of the distances between the plurality of coordinate points on the keyword boundary box and the plurality of coordinate points on the remaining target boundary boxes;

[0034] A module for determining an area to be extracted is configured to sort the distances between the first coordinate point on the keyword boundary box and the second coordinate point on each of the remaining target boundary boxes, and select the target boundary box with the shortest distance and the target text meeting the preset conditions as the area to be extracted that matches the keyword boundary box;

[0035] The extraction module is configured to extract keywords in the keyword boundary box as indicator names, and to extract target text in the area to be extracted as indicator measurement values.

[0036] Further, the multiple directions include upper left, upper, upper right, right, lower right, lower, lower left and left.

[0037] Further, the multiple coordinate points include an upper left corner point, an upper middle point, an upper right corner point, a right middle point, a lower right corner point, a lower middle point, a lower left corner point and a left middle point on the target bounding box.

[0038] Furthermore, the recognition module is further configured to: obtain coordinate values ​​of four corner points of the target bounding box.

[0039] Furthermore, the coordinate point determination module is further configured as follows:

[0040] When the remaining target bounding boxes are located above the keyword bounding box, the upper midpoint of the keyword bounding box is used as the first coordinate point, and the lower midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0041] When the remaining target bounding boxes are located below the keyword bounding box, the lower midpoint of the keyword bounding box is used as the first coordinate point, and the upper midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0042] When the remaining target bounding boxes are located to the left of the keyword bounding box, the left midpoint of the keyword bounding box is used as the first coordinate point, and the right midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0043] When the remaining target bounding boxes are located to the right of the keyword bounding box, the right midpoint of the keyword bounding box is used as the first coordinate point, and the left midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0044] When the remaining target bounding boxes are located at the upper left of the keyword bounding box, the upper left corner point of the keyword bounding box is used as the first coordinate point, and the lower right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0045] When the remaining target bounding boxes are located at the upper right corner of the keyword bounding box, the upper right corner point of the keyword bounding box is used as the first coordinate point, and the lower left corner point of the remaining target bounding boxes is used as the second coordinate point;

[0046] When the remaining target bounding boxes are located at the lower left of the keyword bounding box, the lower left corner point of the keyword bounding box is used as the first coordinate point, and the upper right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0047] When the remaining target bounding boxes are located at the lower right corner of the keyword bounding box, the lower right corner point of the keyword bounding box is used as the first coordinate point, and the upper left corner point of the remaining target bounding boxes is used as the second coordinate point.

[0048] The method and device for extracting the index measurement value of a medical instrument provided by the present invention obtains all target bounding boxes and target texts on the medical instrument image through OCR recognition technology, and then obtains the direction of the index measurement value bounding box based on the original "multi-direction + multi-calculation coordinate point" technology of the present invention, obtains the distance between the index measurement value bounding box and the index name bounding box according to the calculation coordinate point, and finally associates the index name and the index measurement value according to the image correspondence. The above method and device of the present invention can efficiently, conveniently and accurately extract the index name, index measurement value and the correspondence between the two on the medical instrument. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0050] Figure 1 It is a medical instrument screen display photo provided by an embodiment of the present application;

[0051] Figure 2 It is a flow chart of a method for extracting index measurement values ​​of a medical instrument provided by one embodiment of the present application;

[0052] Figure 3 is a schematic diagram of a target bounding box and target text identified by a target detection algorithm and OCR provided by an embodiment of the present application;

[0053] Figure 4It is a schematic diagram of the positional relationship between the keyword of the indicator name, the indicator value and the corresponding bounding box provided by an embodiment of the present application;

[0054] Figure 5 is a schematic diagram of eight directions around a bounding box provided by an embodiment of the present application;

[0055] Figure 6 is a schematic diagram of eight calculated coordinate points on a bounding box provided by an embodiment of the present application;

[0056] Figure 7 is a schematic diagram of the shortest distance of a bounding box provided by an embodiment of the present application;

[0057] Figure 8 It is a structural schematic diagram of a device for extracting index measurement values ​​of a medical instrument provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0060] It should be understood that although the terms first, second, third, etc. may be used to describe the acquisition modules in the embodiments of the present invention, the acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.

[0061] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0062] It should be noted that the directional words such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is formed "on" or "under" another element, it can not only be formed directly "on" or "under" another element, but also be formed "on" or "under" another element indirectly through an intermediate element.

[0063] OCR (Optical Character Recognition) is a technology for searching, extracting and identifying text in images. Extracting and identifying various texts on the front images or screen images of various types of medical instruments through OCR can greatly improve the efficiency and accuracy of collecting indicator data of health monitoring instruments (hereinafter collectively referred to as medical instruments) from different manufacturers and different versions.

[0064] However, see Figure 1 Medical instruments have various types and models, and the correspondence between the indicator names and indicator measurement values ​​that need to be extracted is complicated. OCR recognition can only be used for medical instruments of a single type, a single indicator, and a single extraction logic, and the results can be directly extracted without much subsequent processing; or it can be used to identify multiple types of medical instruments that do not need to extract the correspondence between indicator names and indicator measurement values, that is: there is no need to do more in-depth indicator recognition, just record the entire recognition result, including text and the coordinates of each text, directly enter the information data, and restore and display it on the corresponding system; or it can be used for simple classification of medical instruments, and judge whether the instrument is a blood oximeter or a blood pressure monitor based on the recognized text, without further extraction.

[0065] In order to efficiently, conveniently and accurately extract the indicator name, indicator measurement value and the corresponding relationship between the two on the medical instrument, an embodiment of the present application provides a method for extracting the indicator measurement value of the medical instrument, see Figure 2 , including the following steps:

[0066] Step S101, obtaining a medical instrument image with index measurement values.

[0067] Specifically, based on the actual use of medical equipment, collect pictures of each instrument model, manufacturer, and related screens. If it is a multifunctional medical instrument, collect pictures of the screen corresponding to each function. Based on the actual use of the instrument, clarify the health indicators contained in each function screen of each model of instrument, such as Figure 1As shown, the screen of the blood pressure meter on the left contains health indicators and indicator measurement values ​​such as "high pressure: 128, low pressure: 75, pulse: 88", and the screen of the blood glucose meter on the right contains health indicators and indicator measurement values ​​such as "blood sugar: 5.8". The acquired medical instrument image with indicator measurement values ​​is input into the OCR recognition model.

[0068] Step S102: Perform OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image.

[0069] See also Figure 3 According to the model of the medical instrument, the medical instrument image is subjected to OCR recognition to obtain all the target bounding box (i.e., box) information and the identified target text information in the image. The target bounding box information includes the position coordinates of the four corner points of each rectangular bounding box, and the target text information includes various text information such as numerical values, characters, and words in the medical instrument image. Each target text information is located in the corresponding target bounding box.

[0070] For example, Figure 3 The JSON data format of the medical instrument image after OCR recognition is as follows:

[0071] {"file_name":"899379aa5c1f4f15956f6268b66de746.jpg","total":["125 high pressure mmHg 75 low pressure mmHg 88 pulse beats / min memory start stop"],

[0072] "ocr":[[{"text":"125","eval":0.998465359210968,"position":[[170,81],[322,78],[323,161],[171,164]]},{"text":"High Pressure","eval":0.9980902671813965, "position":[[325,95],[381,92],[382,117],[327,120]]}, {"text":"mmHg","eval":0.9760223031044006,"position":[[330,115],[379,118],[378,136],[329,133]]},{"text":"75","eval ":0.9876474142074585,"position":[[214,162],[319,162],[319,234],[214,234]]},{"text":"Low pressure","eval":0.9921163320541382, "position":[[333,170],[383,170],[383,192],[333,192]]},{"text":"mmHg", "eval":0.8926085829734802,"position":[[335,192],[380,192],[380,207],[335,207]]},{"text":"88","eval":0.9992 350935935974,"position":[[224,241],[325,236],[328,293],[227,297]]},{"text":"pulse","eval":0.6193825602531433, "position": [[332,237],[387,235], [388,261],[334,263]]},{"text":"beats / minute","eval":0.9876857399940491,"position":[[335,261],[372,261],[372,276], [335,276]]},{"text":"memory","eval":0.9998807907104492, "position":[[163,335],[209,335],[209,357],[163,357]]},{"text":"start stop","eval":0.5203587412834167,"position":[[278,317],[335,317],[335,372], [278,372]]}]],"file_type":"IMG"}.

[0073] From the above data, it can be seen that the OCR recognition result identifies in detail how many target bounding boxes there are and the text content in each target bounding box.

[0074] For example: {"text":"High Pressure","eval":0.9980902671813965, "position":[[325,95],[381,92],[382,117],[327,120]]} is the bounding box information of the keyword "High Pressure", where "position" is the image position area of ​​the keyword, and [[325,95], [381,92], [382,117],[327,120]] are the coordinate values ​​of the four corner points of the bounding box.

[0075] For another example: {"text":"125","eval":0.998465359210968, "position":[[170,81],[322,78],[323,161],[171,164]]} is the bounding box information of the indicator measurement value "125" of the "high pressure" keyword, and "position" is the coordinate position of "125" in the medical instrument image.

[0076] See also Figure 4 , the keyword of the indicator name and the indicator measurement value (indicator value) of the sphygmomanometer on the left are not in order, the indicator measurement value (indicator value) is in front, and the keyword of the indicator name is in the back. The blood glucose meter on the right has only one blood glucose indicator, but the other data on the screen are changing, such as date, time, etc. Although it is possible to judge which is the indicator name by the keyword "mmol / L", the OCR recognition result from top to bottom is "590 9-05 9:06 5.8 O mmol / LMC", so it is difficult to judge which is the indicator measurement value (indicator value) from the entire recognized target text. The core problem of the present invention is to match the indicator name and the indicator measurement value (indicator value), similar to making a connection, for example: connect "high pressure" to "125", "low pressure" to "75", "pulse" to "88", etc.

[0077] Step S103, determining a keyword that can represent the name of the indicator to be extracted from the target text, and determining a keyword bounding box from the target bounding box.

[0078] Specifically, in order to extract the indicator measurement value corresponding to a certain indicator name, it is necessary to first determine the indicator name. Figure 4 As shown, the indicator names may exist directly in the medical instrument image, such as Figure 4 The keywords "high pressure" and "low pressure" in the blood pressure meter on the left are the indicator names, but the indicator names do not appear directly in the blood glucose meter on the right. However, we can use the keyword "mmol / L" as the keyword of the indicator name. After determining the keywords, obtain the bounding box information where these keywords are located, recorded as b1.

[0079] Step S104, taking the keyword bounding box as the central area, dividing the peripheral area surrounding the central area into multiple directions, and determining the direction of the remaining target bounding boxes relative to the central area, which direction belongs to one of the multiple directions.

[0080] Specifically, in order to extract the indicator measurement value corresponding to the indicator name, the present invention adopts the original "multi-direction + multi-calculation coordinate point" technology, first obtaining the direction of the indicator measurement value boundary box, and then obtaining the distance between the indicator measurement value boundary box and the indicator name boundary box according to the calculated coordinate point, and finally associating the indicator name and the indicator measurement value according to the image correspondence.

[0081] This step is used to obtain the direction of the metric measurement value bounding box relative to the keyword bounding box. Figure 5 , taking b1 as the center area, rotating one circle clockwise from the upper left corner, the directions around b1 are divided into multiple directions, preferably eight directions, namely, upper left: 0, upper: 1, upper right: 2, right: 3, lower right: 4, lower: 5, lower left: 6, left: 7, represented by 0, 1, 2, 3, 4, 5, 6, 7 respectively. Then determine the direction of the remaining target bounding box b2 outside the keyword bounding box relative to b1, which should be one of the above eight directions.

[0082] Step S105, setting a plurality of coordinate points on the remaining target bounding boxes and the keyword bounding box respectively, and determining a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding box from the plurality of coordinate points according to the direction of the remaining target bounding box relative to the keyword bounding box; wherein the distance between the first coordinate point on the keyword bounding box and the second coordinate point on the remaining target bounding box is the minimum value of the distances between the plurality of coordinate points on the keyword bounding box and the plurality of coordinate points on the remaining target bounding box.

[0083] For details, see Figure 6, define calculation coordinate points on all bounding boxes including the keyword bounding box, and each bounding box rotates clockwise from the upper left corner point for one circle, and set multiple calculation coordinate points respectively, preferably eight calculation coordinate points, namely, the upper left corner point, the upper middle point, the upper right corner point, the right middle point, the lower right corner point, the lower middle point, the lower left corner point, and the left middle point.

[0084] Next, according to the direction of the remaining target bounding box b2 relative to b1, the first coordinate point c1 on the keyword bounding box b1 and the second coordinate point c2 on the remaining target bounding box b2 are determined from the eight calculated coordinate points. The selection of the first coordinate point c1 and the second coordinate point c2 needs to ensure that the distance between c1 and c2 is the shortest distance between the eight calculated coordinate points on b1 and the eight calculated coordinate points on b2, that is, the shortest distance from b1 to b2.

[0085] See also Figure 7 , the following details the selection rules of the two coordinate points c1 and c2:

[0086] When the remaining target bounding box b2 is located above the keyword bounding box b1, the upper midpoint of the keyword bounding box b1 is used as the first coordinate point c1, and the lower midpoint of the remaining target bounding box b2 is used as the second coordinate point c2;

[0087] When the remaining target bounding box b2 is located below the keyword bounding box b1, the lower midpoint of the keyword bounding box b1 is used as the first coordinate point c1, and the upper midpoint of the remaining target bounding box b2 is used as the second coordinate point c2;

[0088] When the remaining target bounding box b2 is located to the left of the keyword bounding box b1, the left midpoint of the keyword bounding box b1 is used as the first coordinate point c1, and the right midpoint of the remaining target bounding box b2 is used as the second coordinate point c2;

[0089] When the remaining target bounding box b2 is located to the right of the keyword bounding box b1, the right midpoint of the keyword bounding box b1 is used as the first coordinate point c1, and the left midpoint of the remaining target bounding box b2 is used as the second coordinate point c2;

[0090] When the remaining target bounding box b2 is located at the upper left of the keyword bounding box b1, the upper left corner point of the keyword bounding box b1 is used as the first coordinate point c1, and the lower right corner point of the remaining target bounding box b2 is used as the second coordinate point c2;

[0091] When the remaining target bounding box b2 is located at the upper right corner of the keyword bounding box b1, the upper right corner point of the keyword bounding box b1 is used as the first coordinate point c1, and the lower left corner point of the remaining target bounding box b2 is used as the second coordinate point c2;

[0092] When the remaining target bounding box b2 is located at the lower left of the keyword bounding box b1, the lower left corner point of the keyword bounding box b1 is used as the first coordinate point c1, and the upper right corner point of the remaining target bounding box b2 is used as the second coordinate point c2;

[0093] When the remaining target bounding box b2 is located at the lower right of the keyword bounding box b1, the lower right corner point of the keyword bounding box b1 is used as the first coordinate point c1, and the upper left corner point of the remaining target bounding box b2 is used as the second coordinate point c2.

[0094] Step S106, sorting the distances between the first coordinate point on the keyword bounding box and the second coordinate point on each of the remaining target bounding boxes, and taking the target bounding box with the shortest distance and the target text meeting the preset conditions as the area to be extracted that matches the keyword bounding box.

[0095] Specifically, the distance d between the first coordinate point c1 on the keyword bounding box b1 and the second coordinate point c2 on each of the remaining target bounding boxes b2 is obtained according to the method of step S105. The distance d can be obtained by calculating the geometric relationship of the coordinates, for example, by calculating the relationship between the side lengths of the triangle:

[0096] d =

[0097] in, is the horizontal coordinate value of the first coordinate point c1, is the horizontal coordinate value of the second coordinate point c2, is the ordinate value of the first coordinate point c1, is the ordinate value of the second coordinate point c2.

[0098] Then, these distances are sorted, and the target bounding box b2 with the shortest distance is used as the candidate region to be extracted. When the candidate region to be extracted meets the preset additional conditions, the candidate region to be extracted is used as the region to be extracted that matches the keyword bounding box. It should be pointed out that the preset additional conditions should be the index features that the text in the candidate region to be extracted must meet, so as to ensure that the text in the candidate region to be extracted must be the index measurement value, for example: the preset additional condition is "text must be a number".

[0099] Step S107 , extracting keywords in the keyword boundary box as indicator names, and extracting target text in the to-be-extracted area as indicator measurement values.

[0100] Specifically, after determining the area to be extracted, the keyword in the keyword boundary box is extracted as the indicator name, and the target text in the area to be extracted is extracted as the indicator measurement value. At this time, the keyword (representing the indicator name) and the indicator measurement value are accurately corresponding. In the same way, the indicator measurement value corresponding to each keyword is obtained respectively to realize the extraction of the indicator measurement value of the medical instrument.

[0101] The method for extracting the index measurement value of the medical instrument provided in the present application obtains all the target bounding boxes and target texts on the medical instrument image through OCR recognition technology, and then obtains the direction of the index measurement value bounding box based on the original "multi-direction + multi-calculation coordinate point" technology of the present invention, obtains the distance between the index measurement value bounding box and the index name bounding box according to the calculation coordinate point, and finally associates the index name and the index measurement value according to the image correspondence. The method of the present invention can efficiently, conveniently and accurately extract the index name, index measurement value and the corresponding relationship between the two of the medical instrument.

[0102] See also Figure 8 Another embodiment of the present invention further provides a device 200 for extracting index measurement values ​​of a medical instrument, comprising an image acquisition module 201, an identification module 202, a determination module 203, a direction determination module 204, a coordinate point determination module 205, a to-be-extracted area determination module 206, and an extraction module 207. The device 200 for extracting index measurement values ​​of a medical instrument can execute each step in the above method embodiment.

[0103] Specifically, the device 200 for extracting the index measurement value of the medical instrument includes:

[0104] An image acquisition module 201 is configured to acquire a medical instrument image with an index measurement value;

[0105] The recognition module 202 is configured to perform OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image;

[0106] A determination module 203 is configured to determine a keyword that can represent the name of the indicator to be extracted from the target text, and determine a keyword bounding box from the target bounding box;

[0107] A direction determination module 204 is configured to take the keyword bounding box as a central area, divide the peripheral area surrounding the central area into multiple directions, and determine the direction of the remaining target bounding boxes relative to the central area, where the direction belongs to one of the multiple directions;

[0108] The coordinate point determination module 205 is configured to set a plurality of coordinate points on the remaining target bounding boxes and the keyword bounding box respectively, and determine a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding box from the plurality of coordinate points according to the direction of the remaining target bounding box relative to the keyword bounding box; wherein the distance between the first coordinate point on the keyword bounding box and the second coordinate point on the remaining target bounding box is the minimum value of the distances between the plurality of coordinate points on the keyword bounding box and the plurality of coordinate points on the remaining target bounding boxes;

[0109] The to-be-extracted region determination module 206 is configured to sort the distances between the first coordinate point on the keyword boundary box and the second coordinate point on each of the remaining target boundary boxes, and select the target boundary box with the shortest distance and the target text meeting the preset condition as the to-be-extracted region matching the keyword boundary box;

[0110] The extraction module 207 is configured to extract keywords in the keyword boundary box as indicator names, and extract target text in the to-be-extracted area as indicator measurement values.

[0111] Among them, the multiple directions include upper left, upper, upper right, right, lower right, lower, lower left and left.

[0112] The multiple coordinate points include an upper left corner point, an upper middle point, an upper right corner point, a right middle point, a lower right corner point, a lower middle point, a lower left corner point and a left middle point on the target bounding box.

[0113] Furthermore, the recognition module 202 is further configured to: obtain coordinate values ​​of four corner points of the target bounding box.

[0114] Furthermore, the coordinate point determination module 205 is further configured to:

[0115] When the remaining target bounding boxes are located above the keyword bounding box, the upper midpoint of the keyword bounding box is used as the first coordinate point, and the lower midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0116] When the remaining target bounding boxes are located below the keyword bounding box, the lower midpoint of the keyword bounding box is used as the first coordinate point, and the upper midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0117] When the remaining target bounding boxes are located to the left of the keyword bounding box, the left midpoint of the keyword bounding box is used as the first coordinate point, and the right midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0118] When the remaining target bounding boxes are located to the right of the keyword bounding box, the right midpoint of the keyword bounding box is used as the first coordinate point, and the left midpoint of the remaining target bounding boxes is used as the second coordinate point;

[0119] When the remaining target bounding boxes are located at the upper left of the keyword bounding box, the upper left corner point of the keyword bounding box is used as the first coordinate point, and the lower right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0120] When the remaining target bounding boxes are located at the upper right corner of the keyword bounding box, the upper right corner point of the keyword bounding box is used as the first coordinate point, and the lower left corner point of the remaining target bounding boxes is used as the second coordinate point;

[0121] When the remaining target bounding boxes are located at the lower left of the keyword bounding box, the lower left corner point of the keyword bounding box is used as the first coordinate point, and the upper right corner point of the remaining target bounding boxes is used as the second coordinate point;

[0122] When the remaining target bounding boxes are located at the lower right corner of the keyword bounding box, the lower right corner point of the keyword bounding box is used as the first coordinate point, and the upper left corner point of the remaining target bounding boxes is used as the second coordinate point.

[0123] It should be noted that the device 200 for extracting the index measurement value of the medical instrument provided in this embodiment corresponds to the technical solution that can be used to execute each method embodiment, and its implementation principle and technical effect are similar to the method, which will not be repeated here.

[0124] The above description is only a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but also should cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A method for extracting index measurement values ​​of a medical instrument, characterized in that: include: Acquire medical instrument images with index measurements; Performing OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image; Determine a keyword that can represent the name of the indicator to be extracted from the target text, and determine a keyword bounding box from the target bounding box; Taking the keyword bounding box as the central area, dividing the peripheral area surrounding the central area into multiple directions, the multiple directions including upper left, upper, upper right, right, lower right, lower, lower left and left, and determining the direction of the remaining target bounding boxes relative to the central area, the direction belonging to one of the multiple directions; A plurality of coordinate points are respectively set on the remaining target bounding boxes and the keyword bounding box, the plurality of coordinate points including an upper left corner point, an upper middle point, an upper right corner point, a right middle point, a lower right corner point, a lower middle point, a lower left corner point and a left middle point on the target bounding box, and a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding box are determined from the plurality of coordinate points according to the direction of the remaining target bounding box relative to the keyword bounding box; wherein the distance between the first coordinate point on the keyword bounding box and the second coordinate point on the remaining target bounding box is the minimum value among the distances between the plurality of coordinate points on the keyword bounding box and the plurality of coordinate points on the remaining target bounding box; sorting the distances between the first coordinate point on the keyword bounding box and the second coordinate point on each of the remaining target bounding boxes, and taking the target bounding box with the shortest distance and the target text meeting the preset conditions as the area to be extracted that matches the keyword bounding box; Keywords in the keyword boundary box are extracted as indicator names, and target text in the to-be-extracted area is extracted as indicator measurement values.

2. The method for extracting the index measurement value of a medical instrument according to claim 1, characterized in that: The target bounding box information includes coordinate values ​​of four corner points of the target bounding box.

3. The method for extracting the index measurement value of a medical instrument according to claim 1, characterized in that: The step of determining a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding boxes from the plurality of coordinate points according to the directions of the remaining target bounding boxes relative to the keyword bounding box comprises: When the remaining target bounding boxes are located above the keyword bounding box, the upper midpoint of the keyword bounding box is used as the first coordinate point, and the lower midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located below the keyword bounding box, the lower midpoint of the keyword bounding box is used as the first coordinate point, and the upper midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located to the left of the keyword bounding box, the left midpoint of the keyword bounding box is used as the first coordinate point, and the right midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located to the right of the keyword bounding box, the right midpoint of the keyword bounding box is used as the first coordinate point, and the left midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the upper left of the keyword bounding box, the upper left corner point of the keyword bounding box is used as the first coordinate point, and the lower right corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the upper right corner of the keyword bounding box, the upper right corner point of the keyword bounding box is used as the first coordinate point, and the lower left corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the lower left of the keyword bounding box, the lower left corner point of the keyword bounding box is used as the first coordinate point, and the upper right corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the lower right corner of the keyword bounding box, the lower right corner point of the keyword bounding box is used as the first coordinate point, and the upper left corner point of the remaining target bounding boxes is used as the second coordinate point.

4. A device for extracting index measurement values ​​of a medical instrument, characterized in that: include: An image acquisition module configured to acquire a medical instrument image with an indicator measurement value; A recognition module is configured to perform OCR recognition on the medical instrument image to obtain all target bounding boxes and target texts in the medical instrument image; A determination module is configured to determine a keyword that can represent the name of the indicator to be extracted from the target text, and determine a keyword bounding box from the target bounding box; a direction determination module configured to take the keyword bounding box as a central area, divide the peripheral area surrounding the central area into a plurality of directions, the plurality of directions including upper left, upper, upper right, right, lower right, lower, lower left and left, and determine a direction of the remaining target bounding boxes relative to the central area, the direction belonging to one of the plurality of directions; A coordinate point determination module is configured to set a plurality of coordinate points on the remaining target bounding boxes and the keyword bounding box respectively, the plurality of coordinate points including an upper left corner point, an upper middle point, an upper right corner point, a right middle point, a lower right corner point, a lower middle point, a lower left corner point and a left middle point on the target bounding box, and determine a first coordinate point on the keyword bounding box and a second coordinate point on the remaining target bounding box from the plurality of coordinate points according to the direction of the remaining target bounding box relative to the keyword bounding box; wherein the distance between the first coordinate point on the keyword bounding box and the second coordinate point on the remaining target bounding box is the minimum value of the distances between the plurality of coordinate points on the keyword bounding box and the plurality of coordinate points on the remaining target bounding box; A module for determining an area to be extracted is configured to sort the distances between the first coordinate point on the keyword boundary box and the second coordinate point on each of the remaining target boundary boxes, and select the target boundary box with the shortest distance and the target text meeting the preset conditions as the area to be extracted that matches the keyword boundary box; The extraction module is configured to extract keywords in the keyword boundary box as indicator names, and to extract target text in the area to be extracted as indicator measurement values.

5. The device for extracting index measurement values ​​of a medical instrument according to claim 4, characterized in that: The recognition module is further configured to: obtain coordinate values ​​of four corner points of the target bounding box.

6. The device for extracting index measurement values ​​of a medical instrument according to claim 4, characterized in that: The coordinate point determination module is further configured as follows: When the remaining target bounding boxes are located above the keyword bounding box, the upper midpoint of the keyword bounding box is used as the first coordinate point, and the lower midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located below the keyword bounding box, the lower midpoint of the keyword bounding box is used as the first coordinate point, and the upper midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located to the left of the keyword bounding box, the left midpoint of the keyword bounding box is used as the first coordinate point, and the right midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located to the right of the keyword bounding box, the right midpoint of the keyword bounding box is used as the first coordinate point, and the left midpoint of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the upper left of the keyword bounding box, the upper left corner point of the keyword bounding box is used as the first coordinate point, and the lower right corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the upper right corner of the keyword bounding box, the upper right corner point of the keyword bounding box is used as the first coordinate point, and the lower left corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the lower left of the keyword bounding box, the lower left corner point of the keyword bounding box is used as the first coordinate point, and the upper right corner point of the remaining target bounding boxes is used as the second coordinate point; When the remaining target bounding boxes are located at the lower right corner of the keyword bounding box, the lower right corner point of the keyword bounding box is used as the first coordinate point, and the upper left corner point of the remaining target bounding boxes is used as the second coordinate point.

Images