Test paper identification method, device, storage medium and equipment

Abstract

The embodiments of the present invention provide a test paper recognition method, device, storage medium and equipment, which belong to the field of image processing technology. The method includes: obtaining an image to be recognized, wherein the image to be recognized includes a test paper image to be recognized; using an image recognition method based on a first deep learning method, extracting the test paper image to be recognized from the image to be recognized; using a preset category model, identifying the category to which the mark in the test paper image to be recognized belongs, and extracting the detection line information corresponding to the category; using an image recognition method based on a second deep learning method, determining the position of the recognition line from the test paper image to be recognized; using a regular image processing method, determining the test paper detection result according to the image brightness value corresponding to the position of the recognition line and the detection line information. The embodiments of the present invention are applicable to the test paper recognition process.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a test paper recognition method, device, storage medium and equipment. Background Art

[0002] Female pregnancy test strips are one of the main means of scientific pregnancy preparation and artificial assisted pregnancy. Female pregnancy test strips mainly include two categories: ovulation test strips and early pregnancy test strips. Ovulation test strips mainly predict when ovulation will occur by detecting the peak level of luteinizing hormone (LH). Early pregnancy test strips mainly determine whether you are pregnant and judge the development of the embryo by detecting human chorionic gonadotropin (HCG). For the above two types of test strips, the yin and yang of the indicators to be tested are currently mainly judged by comparing the color depth of the test line (T) and the control line (C) through human eye observation. When the test line (T) is darker or equal to the control line (C), the test strip shows positive; otherwise, the test strip shows negative, and human eye recognition often has deviations.

[0003] With the rise of mobile Internet, Internet of Things and artificial intelligence, miniaturized data acquisition equipment and test paper image recognition methods based on smart phones have gradually become mainstream. In the prior art, there are generally two recognition methods, one is an image recognition method based on deep learning, using a convolutional neural network, training a deep learning model of an image, extracting the deep learning features of the image and building a database, and finally classifying and identifying the features by SVM (Support Vector Machine); the other is an image recognition method based on regular image information for feature extraction, first performing Harris corner detection to calibrate the image, then converting the image space from RGB to CIELAB, calculating the color difference between the test paper color and the color block of the standard colorimetric card, and then using the nearest neighbor algorithm to classify the test paper color into the category corresponding to the colorimetric card, to complete the analysis of the test paper measurement index. However, for the method of test paper recognition in the above-mentioned prior art, it is impossible to avoid the test paper detection result being affected by a complex background, and affected by a variety of brand test papers, the recognition technology in the prior art cannot achieve universal recognition. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a test paper recognition method, device, storage medium and equipment, which solves the problem in the prior art that the test paper recognition is affected by the complex background, resulting in low accuracy of the test paper detection results, and the recognition technology has poor versatility for multiple types of brand test papers. In the embodiments of the present invention, a test paper recognition method that mixes deep learning and regular image processing is used to extract the test paper image from the image to be recognized, avoiding the influence of the complex background on the test paper image, and then directly determining the position of the recognition line in the test paper image through color jump, thereby improving the recognition versatility of the test paper image.

[0005] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a test paper recognition method, which includes: obtaining an image to be recognized, wherein the image to be recognized includes a test paper image to be recognized; using an image recognition method based on a first deep learning to extract the test paper image to be recognized from the image to be recognized; using a preset category model to identify the category to which a mark in the test paper image to be recognized belongs, and extracting detection line information corresponding to the category; using an image recognition method based on a second deep learning to determine the position of the recognition line from the test paper image to be recognized; using a regular image processing method to determine the test paper detection result according to the pixel brightness value corresponding to the position of the recognition line and the detection line information.

[0006] Furthermore, the method of extracting the test paper image to be identified from the image to be identified by using the image recognition method based on the first deep learning includes: using a preset HED model to obtain an edge grayscale image corresponding to the test paper image to be identified from the image to be identified; and using OpenCV technology to map the test paper image to be identified in the image to be identified according to the edge grayscale image.

[0007] Furthermore, the preset HED model is established in the following manner: preprocessing the image samples to obtain image training samples that meet the specified pixel size; marking the edges of the test paper in the image training samples to obtain image marked samples; using the image training samples and the corresponding image marked samples to train the pre-trained HED model until the loss function value of the pre-trained HED model tends to be stable, and determining that the pre-trained HED model is the preset HED model.

[0008] Furthermore, the preset category model is obtained in the following manner: obtaining a test paper image training set, classifying the test paper images in the test paper image training set according to the category of the mark, and storing the category and its corresponding detection line information; using the test paper image training set and the category corresponding to each test paper image to train the pre-trained CNN model until the loss function value of the pre-trained CNN model tends to be stable, and determining that the pre-trained CNN model is the preset category model.

[0009] Furthermore, the method of using the second deep learning-based image recognition method to determine the position of the identification line from the test paper image to be identified includes: using a preset CRNN model to obtain a region classification in the test paper image to be identified, the region classification including a blank area, an area to the left of the identification line position, and an area to the right of the identification line position; determining the position of the identification line in the test paper image to be identified based on the region classification.

[0010] Furthermore, the method of using the preset CRNN model to obtain the region classification in the test paper image to be identified includes: using the preset CRNN model to convert the test paper image to be identified into feature sequence information; and determining the region classification according to the color jump in the feature sequence information.

[0011] Furthermore, the CRNN model is established in the following manner: preprocessing the test paper image samples to obtain test paper image training samples that meet the set pixel size; marking the region classifications in the test paper image training samples to obtain test paper image labeled samples; using the test paper image training samples and the corresponding test paper image labeled samples to train the pre-trained CRNN model until the loss function value of the pre-trained CRNN model tends to be stable, and determining that the pre-trained CRNN model is the preset CRNN model.

[0012] Furthermore, the method of utilizing regular image processing to determine the test paper detection result according to the pixel brightness value corresponding to the identification line position and the detection line information includes: converting the color space of the identification line position into the LAB color space; extracting the pixel brightness value corresponding to the identification line position in the LAB color space; and determining the test paper detection result according to the comparison result of the pixel brightness value and the detection line information.

[0013] Furthermore, after determining the test paper test result, the method further includes: determining the current physiological information of the user corresponding to the image to be identified based on the test paper test result; and providing the test paper test result and the current physiological information of the user to the user.

[0014] Furthermore, the image to be identified also includes a user identification. After determining the test paper detection result, the method also includes: extracting the user identification included in the image to be identified; searching for historical detection information corresponding to the user identification in a database, the historical detection information including historical test paper detection results and historical physiological information; determining the physiological cycle corresponding to the user based on the historical physiological information and the current physiological information; determining the recommended time for the user to perform the next test paper detection based on the physiological cycle, and providing the recommended time to the user.

[0015] Correspondingly, an embodiment of the present invention also provides a test paper identification device, including: an acquisition unit, used to acquire an image to be identified, the image to be identified including a test paper image to be identified; a test paper image extraction unit, used to extract the test paper image to be identified from the image to be identified by using an image recognition method based on a first deep learning; a category determination unit, used to identify the category to which the mark in the test paper image to be identified belongs by using a preset category model, and extract the detection line information corresponding to the category; a position determination unit, used to determine the position of the identification line from the test paper image to be identified by using an image recognition method based on a second deep learning; and a result determination unit, used to determine the test paper detection result according to the image brightness value corresponding to the position of the identification line and the detection line information by using a regular image processing method.

[0016] Furthermore, the test paper image extraction unit is also used to: use a preset HED model to obtain an edge grayscale map corresponding to the test paper image to be identified from the image to be identified; and use OpenCV technology to map the test paper image to be identified in the image to be identified according to the edge grayscale map.

[0017] Furthermore, the preset HED model is established in the following manner: preprocessing the image samples to obtain image training samples that meet the specified pixel size; marking the edges of the test paper in the image training samples to obtain image marked samples; using the image training samples and the corresponding image marked samples to train the pre-trained HED model until the loss function value of the pre-trained HED model tends to be stable, and determining that the pre-trained HED model is the preset HED model.

[0018] Furthermore, the preset category model is obtained in the following manner: obtaining a test paper image training set, classifying the test paper images in the test paper image training set according to the category of the mark, and storing the category and its corresponding detection line information; using the test paper image training set and the category corresponding to each test paper image to train the pre-trained CNN model until the loss function value of the pre-trained CNN model tends to be stable, and determining that the pre-trained CNN model is the preset category model.

[0019] Furthermore, the position determination unit is also used to: use a preset CRNN model to obtain a region classification in the test paper image to be identified, the region classification including a blank area, an area to the left of the identification line position, and an area to the right of the identification line position; and determine the identification line position in the test paper image to be identified based on the region classification.

[0020] Furthermore, the position determination unit is also used to: use the preset CRNN model to convert the test paper image to be identified into feature sequence information; and determine the region classification according to the color jump in the feature sequence information.

[0021] Furthermore, the CRNN model is established in the following manner: preprocessing the test paper image samples to obtain test paper image training samples that meet the set pixel size; marking the region classifications in the test paper image training samples to obtain test paper image labeled samples; using the test paper image training samples and the corresponding test paper image labeled samples to train the pre-trained CRNN model until the loss function value of the pre-trained CRNN model tends to be stable, and determining that the pre-trained CRNN model is the preset CRNN model.

[0022] Furthermore, the result determination unit is also used to: convert the color space of the identification line position into the LAB color space; extract the image brightness value corresponding to the identification line position in the LAB color space; and determine the test paper detection result based on the comparison result of the image brightness value and the detection line information.

[0023] Furthermore, the device also includes: a physiological information determination unit, used to determine the current physiological information of the user corresponding to the image to be identified based on the test paper detection result; and a processing unit, used to provide the test paper detection result and the current physiological information of the user to the user.

[0024] Furthermore, the image to be identified also includes a user identifier, and the device also includes: an identifier extraction unit, used to extract the user identifier included in the image to be identified; a search unit, used to search for historical detection information corresponding to the user identifier in a database, the historical detection information including historical test paper detection results and historical physiological information; a cycle determination unit, used to determine the physiological cycle corresponding to the user based on the historical physiological information and the current physiological information; the processing unit is also used to determine the recommended time for the user to perform the next test paper detection based on the physiological cycle, and provide the recommended time to the user.

[0025] Correspondingly, an embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the test paper identification method as described above.

[0026] Correspondingly, an embodiment of the present invention also provides a device, which includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the test paper identification method as described above.

[0027] Through the above technical solution, a test paper recognition method that is a mixture of deep learning and regular image processing is used to extract the test paper image from the image to be recognized, thereby avoiding the influence of complex background on the test paper image. Then, the position of the recognition line in the test paper image is directly determined by color jump, thereby determining the test paper detection result according to the image brightness value corresponding to the recognition line position and the detection line information corresponding to the test paper category. The embodiment of the present invention solves the problem in the prior art that the test paper recognition is affected by the complex background, resulting in low accuracy of the test paper detection result, improves the recognition versatility of the test paper image, is convenient and fast, and saves time and cost.

[0028] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0030] Figure 1 It is a flow chart of a test paper identification method provided by an embodiment of the present invention;

[0031] Figure 2 It is a flow chart of another test paper identification method provided by an embodiment of the present invention;

[0032] Figure 3 It is a structural schematic diagram of a test paper identification device provided by an embodiment of the present invention;

[0033] Figure 4 is a structural schematic diagram of another test paper identification device provided by an embodiment of the present invention;

[0034] Figure 5 is a structural schematic diagram of another test paper identification device provided by an embodiment of the present invention;

[0035] Figure 6 It is a structural schematic diagram of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0037] Figure 1 FIG. 1 is a flow chart of a test paper identification method provided by an embodiment of the present invention. Figure 1As shown, the method is applied to a server, and the method comprises the following steps:

[0038] Step 101, obtaining an image to be identified, wherein the image to be identified includes a test paper image to be identified;

[0039] Step 102, extracting the test paper image to be identified from the image to be identified by using the first deep learning-based image recognition method;

[0040] Step 103, using a preset category model, identifying the category to which the mark in the test paper image to be identified belongs, and extracting the detection line information corresponding to the category;

[0041] Step 104, using the second deep learning-based image recognition method to determine the position of the recognition line from the test paper image to be recognized;

[0042] Step 105 , using a regular image processing method, determining a test paper detection result according to the image brightness value corresponding to the position of the identification line and the detection line information.

[0043] Among them, the test strips identified by the embodiment of the present invention include POCT (point-of-care testing) test strips, for example, pregnancy test strips, ovulation test strips, multi-test strips, multiple T-line test strips, etc.

[0044] After the user uses the test paper for testing, he can use a device with a camera function to take a photo of the test paper, and upload the image to be identified obtained by the photo to the server to complete the identification of the test paper. In this way, the user only needs to save the image to be identified after the test is completed, without saving the test paper itself, which provides great convenience for the user.

[0045] Since the image to be identified obtained by the user through taking a photo includes not only the image of the test paper to be identified, but also other unnecessary complex background information, therefore, in the embodiment of the present invention, after the server receives the image to be identified including the image of the test paper to be identified uploaded by the user, it is necessary to extract the image to be identified from the complex background.

[0046] In addition, due to the particularity of the test paper image to be identified, unlike other captured images, the test paper image to be identified is an elongated strip image. Since the aspect ratio is too large, the traditional image segmentation algorithm cannot be used. Therefore, the embodiment of the present invention uses a preset HED model in step 102 to obtain the edge grayscale image corresponding to the test paper image to be identified from the image to be identified, and uses OpenCV technology to map the test paper image to be identified in the image to be identified according to the edge grayscale image.

[0047] The preset HED model is established in the following way:

[0048] First, the image samples are preprocessed to obtain image training samples that meet the specified pixel size. For example, the specified pixel size is 128*128 pixels. If the image sample resolution exceeds this resolution, it is cropped; if it is less than this resolution, it is stretched and tiled.

[0049] Then, the test paper edge in the image training sample is marked to obtain an image marked sample. The marking method can be marked by using marking software, for example, obtaining the positions of the four vertices of the rectangular test paper edge, mapping the four coordinates into a quadrilateral, and then creating a binary all-black image, and setting the corresponding edges of the quadrilateral to white.

[0050] Afterwards, the pre-trained HED model is trained using the image training samples and the corresponding image labeled samples until the loss function value of the pre-trained HED model tends to be stable, and the pre-trained HED model is determined to be the preset HED model.

[0051] The pre-trained HED model includes 5 convolution layers, and after the image training sample passes through the convolution layer, 5 layers of feature maps are obtained, and the dimensions of each layer of feature maps are shown in Table 1 below. Afterwards, the 5 layers of feature maps are deconvolved into 256*256*1 feature maps respectively, and the 5 deconvolved feature maps are merged to obtain a feature map corresponding to the image training sample, and the merged feature map is compared with the image labeled sample corresponding to the image training sample to obtain the loss function value. After continuous training and optimization, for example, after 100,000 iterations, the loss function value tends to be stable, and the pre-trained HED model at this time is determined as the preset HED model. The model optimizer selects stochastic gradient descent, the learning rate is set to 0.001, and the loss function is the cross entropy function.

[0052] Table 1

[0053] Number of layers Feature map generated after convolution filter 1 256*256*5 2 128*128*14 3 74*74*40 4 37*37*92 5 18*18*196

[0054] By using the preset HED model and OpenCV technology, the test paper image to be identified is extracted from the image to be identified, thereby avoiding the influence of complex background.

[0055] Afterwards, in step 103, since there are multiple brands of multi-link test strips and multi-T line test strips, in order to distinguish the test data represented by each identification line on the test strip, the logo on each test strip is used to distinguish. Because the logos on test strips of different brands are different, the logos on test strips of the same brand containing different indicators are also different. Therefore, a preset category model can be used to identify the category to which the logo in the test strip image to be identified belongs, and to extract the detection line information corresponding to the category. First, it is necessary to establish and implement a preset category model in advance, obtain a test strip image training set, classify the test strip images in the test strip image training set according to the category of the logo, and store the category and its corresponding detection line information in a category database, wherein the detection line information includes the indicator corresponding to the identification line on the test strip of the category and the preset range of the indicator. For example, when the category is C, T line test strips, the detection line information also includes a preset ratio range of the brightness values ​​corresponding to the T line and the C line. Afterwards, the pre-trained CNN model is trained using the test paper image training set and the category corresponding to each test paper image. The model training process is to go through 5 convolutional layers, then perform full connection, and then perform classification until the loss function value of the pre-trained CNN model tends to be stable. The pre-trained CNN model is determined to be the preset category model.

[0056] At the same time, in step 104, the preset CRNN model is used to obtain the regional classification in the test paper image to be identified, and the position of the identification line in the test paper image to be identified is determined according to the regional classification. Specifically, the preset CRNN model is used to convert the test paper image to be identified into feature sequence information, and the regional classification is determined according to the color jump in the feature sequence information, wherein the regional classification includes a blank area, an area to the left of the identification line position, and an area to the right of the identification line position.

[0057] The CRNN model is established in the following way:

[0058] First, the test paper image samples are preprocessed to obtain test paper image training samples that meet the set pixel size. For example, after the test paper image samples are cropped or stretched and tiled, a test paper image training sample of size 1080*96 is obtained. Then, the regional classification in the test paper image training sample is marked to obtain a test paper image marked sample. For example, the regional classification in the test paper image training sample is marked as a blank area None, an area on the left side of the identification line position I-left, and an area on the right side of the identification line position I-right. Of course, for a test paper containing multiple identification lines, the regional classification will also include multiple areas on the left side of the identification line position and areas on the right side of the identification line position. Afterwards, the pre-trained CRNN model is trained using the test paper image training samples and the corresponding test paper image marked samples until the loss function value of the pre-trained CRNN model tends to be stable, and the pre-trained CRNN model is determined to be the preset CRNN model.

[0059] Among them, the test paper can be regarded as a piece of sequence information, from right to left, they are the brand logo area and the identification line area, therefore, the test paper image can be regarded as data with sequence information, and the sequence position of the identification line is predicted by the pre-trained CRNN model. The pre-trained CRNN model is divided into three parts, the first part is a convolution structure, which extracts features in the vertical direction through a three-layer convolutional neural network, and ensures that the scale of the input and output of the convolution structure in the horizontal direction does not change, and converts the test paper image training sample into feature sequence information. The second part is a three-layer bidirectional recurrent neural network structure, where the recurrent neural network unit we select is a GRU structure, because GRU is better than traditional RNN and LSTM in terms of calculation speed and training effect. The third part of the model is a fully connected layer, which performs regional classification on the features output by different sequence coordinates in the recurrent neural network structure, and compares the predicted regional classification with the regional classification in the corresponding test paper image label sample to obtain the loss function value. After continuous training and optimization, for example, after 200 iterations, the loss function value tends to be stable, and the pre-trained CRNN model at this time is determined as the preset CRNN model. Among them, the model optimizer selects stochastic gradient descent, the learning rate is set to 0.001, and the loss function is the cross entropy function.

[0060] After the identification line position is determined through the above steps, in step 105, the color space of the identification line position is converted into the LAB color space. Then, the image brightness value corresponding to the identification line position in the LAB color space is extracted and compared with the brightness value range in the detection line information. For example, when the test paper is a multi-link test paper or a multi-T line test paper, the index corresponding to each identification line on the test paper and the preset range of the index can be determined from the corresponding detection line information, so that the detection result of the index is determined by the value corresponding to the image brightness value within the corresponding preset range. When the test paper is a C or T line test paper, the identification line position is the T line and C line position. After the image brightness value corresponding to the T line and the C line is obtained, the test paper detection result corresponding to the ratio is determined according to the ratio of the image brightness value corresponding to the T line and the C line, and the preset ratio range in the detection line information corresponding to the category of test paper.

[0061] Through the embodiment of the present invention, the image of the test paper to be identified is extracted from the image to be identified using the image recognition method based on the first deep learning, avoiding the influence of complex background on the test paper detection result. Since all brands of test papers have logos, the category to which the logo on the test paper belongs is identified using a preset category model, and the detection line information corresponding to the category is obtained. The position of the identification line is directly determined using the image recognition method based on the second deep learning, and the regular image processing method is used to compare the image brightness value corresponding to the position of the identification line with the detection line information to determine the test paper detection result. The embodiment of the present invention uses a server to quickly identify the test paper detection result, which is convenient and fast, saving time and cost.

[0062] In order to facilitate understanding of the embodiments of the present invention, Figure 2 is a flow chart of a test strip identification method provided by an embodiment of the present invention. Among them, the embodiment of the present invention is described by taking an ovulation test strip as an example. Figure 2 As shown, the following steps are included:

[0063] Step 201: the smart terminal captures the image of the ovulation test paper to be identified, obtains the image to be identified, and uploads the image to be identified to the server, wherein the image to be identified includes the user identification and the image of the ovulation test paper to be identified.

[0064] After the user uses the ovulation test strip for testing, he can use a smart terminal with a camera function to take a photo of the ovulation test strip, and upload the image to be identified to the server to obtain the test strip test result. Therefore, the user only needs to save the image to be identified after the test is completed, and there is no need to save the test strip itself, which provides great convenience for the user.

[0065] Step 202: The server receives the image to be identified uploaded by the smart terminal;

[0066] Step 203, using the preset HED model, obtaining an edge grayscale image corresponding to the test paper image to be identified from the image to be identified;

[0067] Step 204, using OpenCV technology, mapping the edge grayscale map in the image to be identified to obtain the test paper image to be identified;

[0068] Step 205, using a preset category model, identifying the category to which the mark in the test paper image to be identified belongs, and extracting the detection line information corresponding to the category, wherein the detection line information indicates that the category is an ovulation test paper and includes a preset ratio range corresponding to the ratio of the image brightness values ​​of the T line and the C line;

[0069] Step 206, using the preset CRNN model, converting the test paper image to be identified into feature sequence information;

[0070] Step 207, determining the region classification according to the color jump in the characteristic sequence information, the region classification including a blank region, a region to the left of the identification line position, and a region to the right of the identification line position;

[0071] Step 208, determining the position of the identification line in the ovulation test paper image to be identified according to the region classification;

[0072] Step 209, converting the color space of the identification line position into the LAB color space;

[0073] Step 210, extracting the image brightness values ​​corresponding to the C line and T line positions in the LAB color space in the identification line positions, and obtaining the ratio of the image brightness value corresponding to the T line position to the image brightness value corresponding to the C line position;

[0074] Step 211, determining the ovulation test paper test result corresponding to the ratio according to the ratio and the preset ratio range;

[0075] Step 212, determining the current physiological information of the user corresponding to the image to be identified according to the test paper detection result;

[0076] Step 213, extracting the user identification included in the image to be identified;

[0077] Step 214, searching the database for historical test information corresponding to the user identifier, the historical test information including historical ovulation test paper test results and historical physiological information;

[0078] Step 215, determining the physiological cycle corresponding to the user, for example, the ovulation cycle corresponding to the user, according to the historical physiological information and the current physiological information;

[0079] Step 216, determining a recommended time for the user to perform the next ovulation test according to the physiological cycle;

[0080] Step 217: providing the test strip test result, the user's current physiological information and the recommended time to the user.

[0081] For example, the user's corresponding ovulation test strip test results can be used to determine the user's corresponding ovulation cycle, and combined with the current physiological information obtained from the current test strip test results, a comprehensive analysis suggestion is given, such as when the next ovulation test strip test should be performed. In addition, the intelligent terminal can also display the extracted ovulation test strip image to be identified, which is convenient for the user to check and compare.

[0082] Through the embodiments of the present invention, the use of a smart terminal to quickly detect ovulation test strips makes the method of detecting ovulation more convenient, simple, time-saving and cost-effective, allowing users to complete the test without leaving home and understand their own physical condition, without the need for additional expensive instruments, which is convenient and fast, saving time and money, and also helps to better protect the user's personal privacy. In addition, the embodiments of the present invention can quickly and accurately read the test strip test results, converting the test strip test from traditional qualitative detection to quantitative analysis, and provide users with recommended test times, so that users can predict the best time for conception or pregnancy.

[0083] Of course, the embodiment of the present invention is also applicable to the result detection of other multi-test strips, multiple T-line test strips, and one C-line test strip.

[0084] Accordingly, Figure 3 Schematic diagram of the structure of a test paper identification device provided by an embodiment of the present invention. Figure 3 As shown, the device is applied to a server, and the device includes: an acquisition unit 31, used to acquire an image to be identified, wherein the image to be identified includes a test paper image to be identified; a test paper image extraction unit 32, used to extract the test paper image to be identified from the image to be identified by using an image recognition method based on a first deep learning; a category determination unit 33, used to identify the category to which the mark in the test paper image to be identified belongs by using a preset category model, and extract the detection line information corresponding to the category; a position determination unit 34, used to determine the position of the identification line from the test paper image to be identified by using an image recognition method based on a second deep learning; a result determination unit 35, used to determine the test paper detection result according to the image brightness value corresponding to the position of the identification line and the detection line information by using a regular image processing method.

[0085] Furthermore, the test paper image extraction unit is also used to: use a preset HED model to obtain an edge grayscale map corresponding to the test paper image to be identified from the image to be identified; and use OpenCV technology to map the test paper image to be identified in the image to be identified according to the edge grayscale map.

[0086] Furthermore, the preset HED model is established in the following manner: preprocessing the image samples to obtain image training samples that meet the specified pixel size; marking the edges of the test paper in the image training samples to obtain image marked samples; using the image training samples and the corresponding image marked samples to train the pre-trained HED model until the loss function value of the pre-trained HED model tends to be stable, and determining that the pre-trained HED model is the preset HED model.

[0087] Furthermore, the preset category model is obtained in the following manner: obtaining a test paper image training set, classifying the test paper images in the test paper image training set according to the category of the mark, and storing the category and its corresponding detection line information; using the test paper image training set and the category corresponding to each test paper image to train the pre-trained CNN model until the loss function value of the pre-trained CNN model tends to be stable, and determining that the pre-trained CNN model is the preset category model.

[0088] Furthermore, the position determination unit is also used to: use a preset CRNN model to obtain a region classification in the test paper image to be identified, the region classification including a blank area, an area to the left of the identification line position, and an area to the right of the identification line position; and determine the identification line position in the test paper image to be identified based on the region classification.

[0089] Furthermore, the position determination unit is also used to: use the preset CRNN model to convert the test paper image to be identified into feature sequence information; and determine the region classification according to the color jump in the feature sequence information.

[0090] Furthermore, the CRNN model is established in the following manner: preprocessing the test paper image samples to obtain test paper image training samples that meet the set pixel size; marking the region classifications in the test paper image training samples to obtain test paper image labeled samples; using the test paper image training samples and the corresponding test paper image labeled samples to train the pre-trained CRNN model until the loss function value of the pre-trained CRNN model tends to be stable, and determining that the pre-trained CRNN model is the preset CRNN model.

[0091] Furthermore, the result determination unit is also used to: convert the color space of the identification line position into the LAB color space; extract the image brightness value corresponding to the identification line position in the LAB color space; and determine the test paper detection result based on the comparison result of the image brightness value and the detection line information.

[0092] Furthermore, if Figure 4 As shown, the device also includes: a physiological information determination unit 41, which is used to determine the current physiological information of the user corresponding to the image to be identified based on the test paper detection result; a processing unit 42, which is used to provide the test paper detection result and the current physiological information of the user to the user.

[0093] Furthermore, the image to be identified also includes a user identifier, such as Figure 5 As shown, the device also includes: an identification extraction unit 51, used to extract the user identification included in the image to be identified; a search unit 52, used to search the database for historical detection information corresponding to the user identification, the historical detection information including historical test paper detection results and historical physiological information; a cycle determination unit 53, used to determine the physiological cycle corresponding to the user based on the historical physiological information and the current physiological information; the processing unit is also used to determine the recommended time for the user to perform the next test paper detection based on the physiological cycle, and provide the recommended time to the user.

[0094] Through the embodiment of the present invention, the image recognition method based on deep learning is combined with the regular image processing method, and the test paper image to be identified is extracted from the image to be identified, avoiding the influence of complex background on the test paper detection result. Since all brands of test papers have logos, the category to which the logo on the test paper belongs is identified using a preset category model, and the detection line information corresponding to the category is obtained. After determining the position of the identification line, the image brightness value corresponding to the position of the identification line is compared with the detection line information to determine the test paper detection result. The embodiment of the present invention uses a server to quickly identify the test paper detection result, which is convenient and fast, saving time and cost.

[0095] The operation process of this device refers to the implementation process of the above-mentioned test paper identification method.

[0096] Correspondingly, an embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the test paper identification method as described above.

[0097] Accordingly, Figure 6 The present invention also provides a schematic diagram of a device structure. Figure 6As shown, the device 60 includes at least one processor 61, and at least one memory 62 and a bus 63 connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the test paper identification method described in the above embodiment.

[0098] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0100] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0101] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0104] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A test paper identification method, characterized in that: include: Acquire an image to be identified, wherein the image to be identified includes a test paper image to be identified; Extracting the test paper image to be identified from the image to be identified by using an image recognition method based on a first deep learning method, wherein the first deep learning method includes a preset HED model and OpenCV technology; Using a preset category model, identifying the category to which the mark in the test paper image to be identified belongs, and extracting the detection line information corresponding to the category, the preset category model being a pre-trained CNN model; Using the second deep learning-based image recognition method, determining the position of the recognition line from the test paper image to be recognized; Using a regular image processing method, the test paper detection result is determined according to the image brightness value corresponding to the position of the identification line and the detection line information; The method of using the second deep learning-based image recognition method to determine the position of the recognition line from the test paper image to be recognized includes: Using a preset CRNN model, converting the test paper image to be identified into feature sequence information; According to the color jump in the characteristic sequence information, the region classification in the test paper image to be identified is obtained, and the region classification includes a blank region, a region to the left of the identification line position, and a region to the right of the identification line position; Determining the position of the identification line in the to-be-identified test paper image according to the area classification; The extracting the test paper image to be identified from the image to be identified by using the first deep learning-based image recognition method comprises: Using the preset HED model, obtaining an edge grayscale image corresponding to the test paper image to be identified from the image to be identified; By using OpenCV technology, the test paper image to be identified is obtained by mapping the edge grayscale map in the image to be identified.

2. The test paper identification method according to claim 1, characterized in that: The preset HED model is established in the following way: Preprocess the image samples to obtain image training samples that meet the specified pixel size; Marking the edges of the test paper in the image training sample to obtain an image marked sample; The pre-trained HED model is trained using the image training samples and the corresponding image labeled samples until the loss function value of the pre-trained HED model tends to be stable, and the pre-trained HED model is determined to be the preset HED model.

3. The test paper identification method according to claim 1, characterized in that: The preset category model is obtained in the following way: Acquire a test paper image training set, classify the test paper images in the test paper image training set according to the categories of the marks, and store the categories and the corresponding detection line information; The pre-trained CNN model is trained using the test paper image training set and the category corresponding to each test paper image until the loss function value of the pre-trained CNN model tends to be stable, and the pre-trained CNN model is determined to be the preset category model.

4. The test paper identification method according to claim 1, characterized in that: The CRNN model is established in the following way: Preprocess the test paper image samples to obtain test paper image training samples that meet the set pixel size; Classify and mark the regions in the test paper image training sample to obtain a test paper image marked sample; The pre-trained CRNN model is trained using the test paper image training samples and the corresponding test paper image labeled samples until the loss function value of the pre-trained CRNN model tends to be stable, and the pre-trained CRNN model is determined to be the preset CRNN model.

5. The test paper identification method according to claim 1, characterized in that: The method of using a regular image processing method to determine the test paper detection result according to the image brightness value corresponding to the identification line position and the detection line information includes: Convert the color space of the identification line position into LAB color space; Extracting the image brightness value corresponding to the position of the identification line in the LAB color space; The test paper detection result is determined according to a comparison result between the image brightness value and the detection line information.

6. The test paper identification method according to claim 1, characterized in that: After determining the test paper test result, the method further includes: Determining the current physiological information of the user corresponding to the image to be identified according to the test paper detection result; The test strip test result and the user's current physiological information are provided to the user.

7. The test paper identification method according to claim 6, characterized in that: The image to be identified also includes a user identifier. After determining the test paper detection result, the method further includes: Extracting a user identifier included in the image to be identified; Searching for historical test information corresponding to the user identifier in a database, the historical test information including historical test paper test results and historical physiological information; Determining a physiological cycle corresponding to the user according to the historical physiological information and the current physiological information; According to the physiological cycle, a recommended time for the user to perform a next test strip test is determined, and the recommended time is provided to the user.

8. A test paper identification device, characterized in that: include: An acquisition unit, used for acquiring an image to be identified, wherein the image to be identified includes a test paper image to be identified; A test paper image extraction unit, used for extracting the test paper image to be identified from the image to be identified by using an image recognition method based on a first deep learning, wherein the first deep learning includes a preset HED model and OpenCV technology; A category determination unit, used to identify the category to which the mark in the test paper image to be identified belongs by using a preset category model, and extract the detection line information corresponding to the category, wherein the preset category model is a pre-trained CNN model; A position determination unit, used to determine the position of the identification line from the test paper image to be identified by using the second deep learning-based image recognition method; A result determination unit, used to determine the test paper detection result according to the image brightness value corresponding to the position of the identification line and the detection line information by using a regular image processing method; The position determination unit is also used to: convert the test paper image to be identified into feature sequence information using a preset CRNN model; According to the color jump in the characteristic sequence information, the region classification in the test paper image to be identified is obtained, and the region classification includes a blank region, a region to the left of the identification line position, and a region to the right of the identification line position; Determining the position of the identification line in the to-be-identified test paper image according to the area classification; The test paper image extraction unit is also used for: Using a preset HED model, an edge grayscale image corresponding to the test paper image to be identified is obtained from the image to be identified; By using OpenCV technology, the test paper image to be identified is obtained by mapping the edge grayscale map in the image to be identified.

9. The test paper identification device according to claim 8, characterized in that: The preset HED model is established in the following way: Preprocess the image samples to obtain image training samples that meet the specified pixel size; Marking the edges of the test paper in the image training sample to obtain an image marked sample; The pre-trained HED model is trained using the image training samples and the corresponding image labeled samples until the loss function value of the pre-trained HED model tends to be stable, and the pre-trained HED model is determined to be the preset HED model.

10. The test paper identification device according to claim 8, characterized in that: The preset category model is obtained in the following way: Acquire a test paper image training set, classify the test paper images in the test paper image training set according to the categories of the marks, and store the categories and the corresponding detection line information; The pre-trained CNN model is trained using the test paper image training set and the category corresponding to each test paper image until the loss function value of the pre-trained CNN model tends to be stable, and the pre-trained CNN model is determined to be the preset category model.

11. The test paper identification device according to claim 8, characterized in that: The CRNN model is established in the following way: Preprocess the test paper image samples to obtain test paper image training samples that meet the set pixel size; Classify and mark the regions in the test paper image training sample to obtain a test paper image marked sample; The pre-trained CRNN model is trained using the test paper image training samples and the corresponding test paper image labeled samples until the loss function value of the pre-trained CRNN model tends to be stable, and the pre-trained CRNN model is determined to be the preset CRNN model.

12. The test paper identification device according to claim 8, characterized in that: The result determination unit is also used for: Convert the color space of the identification line position into LAB color space; Extracting the image brightness value corresponding to the position of the identification line in the LAB color space; The test paper detection result is determined according to a comparison result between the image brightness value and the detection line information.

13. The test paper identification device according to claim 8, characterized in that: The device also includes: A physiological information determination unit, used to determine the current physiological information of the user corresponding to the image to be identified according to the test paper detection result; A processing unit is used to provide the test paper detection result and the user's current physiological information to the user.

14. The test paper identification device according to claim 13, characterized in that: The image to be identified also includes a user identifier, and the device also includes: An identification extraction unit, used to extract a user identification included in the image to be identified; A search unit, used to search the database for historical test information corresponding to the user identifier, wherein the historical test information includes historical test paper test results and historical physiological information; a cycle determination unit, configured to determine a physiological cycle corresponding to the user according to the historical physiological information and the current physiological information; The processing unit is further used to determine, according to the physiological cycle, a recommended time for the user to perform a next test strip test, and provide the recommended time to the user.

15. A machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the test paper identification method described in any one of claims 1 to 7.

16. A device, characterized in that The device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the test paper identification method described in any one of claims 1-7 above.

Images