Automatic identification method and software for multi-directional and multi-scale COVID-19 antigen detection results

Abstract

The present invention discloses a method for automatically distinguishing the results of multi-directional and multi-scale new coronavirus antigen detection. The method constructs a priori structural elements, performs a priori flexible morphological operator filtering on the original image of the antigen test kit test result, calculates the a priori flexible morphological gradient, extracts the edge of the antigen test kit, extracts the test paper area to be distinguished, eliminates interference such as reflective areas, and locates potential areas in sequence to perform pre-detection of letter discrimination areas; performs discrimination letter modeling in the angle dimension to form a training set, combines samples into a tensor form to form a sample feature matrix; converts the RGB color image to the HSV color space, expands the letter image block to extract features, and constructs a feature matching operator to classify the letters C and T; performs color histogram statistics on the test paper area to be distinguished, calculates the color segmentation threshold, and detects the red discrimination mark; and splices the letter classification result and the red discrimination mark detection result to give the discrimination result of the antigen test.

Description

Technical Field

[0001] The present invention relates to the technical field of novel coronavirus antigen image detection, and specifically, to a method and software for automatically distinguishing multi-directional and multi-scale novel coronavirus antigen detection results. Background Art

[0002] The COVID-19 antigen test is flexible and convenient and can be self-tested at home, so the COVID-19 antigen test has become one of the most commonly used means to effectively screen for COVID-19 infection.

[0003] However, due to the large population base, manually tallying antigen test results incurs significant labor and time costs. Furthermore, manual interpretation of positive results is not suitable, and false negative results can lead to cross-infection. Therefore, automated interpretation of antigen test results is necessary. Existing software typically requires manual entry of test results and lacks the ability to automatically identify results.

[0004] At the same time, due to the influence of factors such as shooting angle, lighting, and distance, it is extremely difficult to automatically distinguish the antigen time results. This application uses prior flexible morphological operations to eliminate the influence of uneven background, multi-feature rectangular area positioning, adaptive color histogram test paper area result extraction, multi-directional sample training and feature matrix generation, and construction of feature matching operator detection result recognition and other processing procedures to achieve automatic discrimination of multi-directional and multi-scale new crown antigen test results. It has the characteristics of simple calculation, easy implementation, and strong robustness, and is suitable for promotion and application. Summary of the Invention

[0005] In response to the defects in the prior art, the purpose of the present invention is to provide a multi-directional and multi-scale method for automatically distinguishing the results of COVID-19 antigen detection, comprising the following steps: capturing the original image of the antigen test kit test result, filtering the test result image with a priori flexible morphological operators to remove noise interference, and performing a priori flexible morphological gradient operations to extract the edges of the antigen test kit;

[0006] Through the comprehensive judgment of quasi-rectangle detection and shape aspect ratio, the area to be judged on the test paper is extracted and the interference of the reflective area is eliminated;

[0007] Use the slider to locate the potential area and calculate the information entropy value to pre-detect the letter discrimination area;

[0008] Model the C and T letters in the angle dimension to form a training set, combine the samples into a tensor form, and extract the expanded sample feature matrix;

[0009] Convert the RGB color image to the HSV color space, extract the letter image blocks to be recognized in the optimal brightness space V, expand the letter image blocks to extract features, and construct a feature matching operator to classify the letters C and T;

[0010] Performing color histogram statistics on the area to be judged on the test paper, adaptively calculating the color segmentation threshold, and detecting the red judgment mark;

[0011] The letter classification results and the red discrimination mark detection results are spliced ​​to give the results of the antigen test paper test.

[0012] In one possible implementation, prior structure elements at 0°, 45°, 90°, and 135° are constructed, and the image of the antigen test kit test result is filtered using a priori flexible morphological operators to remove noise interference. Furthermore, the antigen test kit edge is detected using a priori flexible morphological gradient operations. The steps include:

[0013] The flexible morphological closing and opening operations are performed sequentially through the combination of 0°, 90° and 45°, 135° structural elements respectively;

[0014]

[0015]

[0016] f is the input image, · represents the closing operation, o represents the opening operation, A1, A2, A3, A4 represent the kernels of the prior structure elements, B1, B2, B3, B4 are the soft boundaries of the corresponding structure elements, k is the number of repetitions, g1 is the result of the closing and opening operations of the prior structure elements A1 and A2, and g2 is the result of the closing and opening operations of the prior structure elements A3 and A4;

[0017] Perform dilation and erosion operations on g1 and g2 respectively to obtain the flexible morphological gradient:

[0018]

[0019]

[0020] G=G(g1)+G(g2)

[0021] G(g1) and G(g2) are the morphological gradients of g1 and g2 respectively. is the dilation operation, is the corrosion operation, and G is the edge detection result of the antigen test kit.

[0022] In one possible implementation, the rectangular frame detection is performed by combining quasi-rectangle detection and shape aspect ratio determination, and the steps include:

[0023] Step S1: Extract the number of pixels of the length L and width M of the rectangle, and extract the area to be judged by calculating the aspect ratio R. The calculation formula of R is: R = L / M, where the range of R is (5.5, 6.5);

[0024] The rectangle-like index feature is used to describe the degree of approximation between the detection area and the rectangle. The calculation formula of the rectangle-like index Cmtr is:

[0025] Cmtr=(L*M) / N A

[0026] where N A is the number of pixels contained in the potential area of ​​the test paper, and the value range of Cmtr is (0.9, 1.1);

[0027] When both the aspect ratio and the rectangular index characteristics are met, it is judged as the test paper discrimination area, and the interference of reflective areas and other shapes is eliminated;

[0028] Step S2: The test paper discrimination area is expanded to a matrix of L*M on both sides. In this matrix, N*M letter discriminations are extracted in sequence by sliding the slider. The slider step size St can be adjusted. The default value is St = 0.25L, forming four letter discrimination areas (R1, R2, R3, R4);

[0029] Calculate the entropy values ​​of R1, R2, R3, and R4 in sequence, H(R i ),i=1,2,3,4,

[0030]

[0031] Where i = 1, 2, 3, 4, n = N*M, p j is the probability of pixel j appearing;

[0032] The letter discrimination area is Max((H(R1)+H(R2)),(H(R3)+H(R4))).

[0033] In one possible implementation, the steps of modeling the C and T letters in the angle dimension to form a training set, then combining the samples into a tensor form, and extracting the expanded sample feature matrix include:

[0034] Expand the sample sets of letters C and T to adapt to the recognition of letters C and T under different conditions;

[0035] The C and T letter samples are rotated 360 degrees, and the rotation interval angle is set to a, where a is 15 degrees. The modeling training has a total of 24 angles, generating a sample set, and the sample set has 48 feature vectors;

[0036] The size of each sample is 16*16, and the features are extracted after the samples are expanded, and the feature dimension is 256. The principal components are obtained by PCA transformation, and the first 40 eigenvalues ​​are taken to form a 48*40 sample feature matrix MF.

[0037] In one possible implementation, the step of classifying the letters C and T further includes:

[0038] Step 1: Convert the RGB color image to the HSV color space, use the optimal brightness space V to describe the color change variable, and let the RGB values ​​be Vr, Vg, and Vb respectively;

[0039] When Vr>Vg>Vb or Vr>Vb>Vg, V=Vr;

[0040] When Vb>Vg>Vr or Vb>Vr>Vg, V=Vb;

[0041] When Vg>Vr>Vb or Vg>Vb>Vr, V=Vg;

[0042] Extract the letter image block to be detected from the image of the optimal brightness space V, use PCA transformation to obtain its principal component, take the first 40 eigenvalues, and form a vector with a feature dimension of 40

[0043] Step 2: Construct feature matching operator S is the covariance matrix of MF, is the feature vector of the letter image block to be detected,

[0044] That is the corresponding matching result, the corresponding This is the letter recognition result.

[0045] In one possible implementation, the detection of the red identification mark is determined as follows:

[0046] Red=(R>T1)&(G>T2)&(G<T3)&(B> T4);

[0047] Among them, the values ​​of T1, T2, T3, and T4 are 140, 30, 150, and 20 respectively.

[0048] In one possible implementation, the letter classification result and the red identification mark detection result are spliced ​​together, and the result is identified by a table lookup method. There are four types of results, namely:

[0049] C has result, T has no result, negative;

[0050] C has result, T has result, positive;

[0051] C has no result, T has result, invalid;

[0052] C has results, T has no results, invalid.

[0053] On the other hand, the present application provides an automatic discrimination software for multi-directional and multi-scale new coronavirus antigen test results. The automatic discrimination software is installed and executed in the form of computer and mobile phone software. It can automatically judge the results of the antigen test based on the original image of the antigen test paper test results and display negative or positive on the computer and mobile phone screens based on the results.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The method disclosed in the present invention makes full use of technologies such as flexible morphology and geometric features of prior structural elements, gradually locates the test paper detection area, performs feature extraction at multiple levels after modeling, constructs a multi-directional training set for training, and extracts its main feature matrix; adopts color discrimination and letter discrimination, and finally gives the discrimination result through splicing discrimination of color and letter. The method is simple in calculation, low in complexity, has strong resistance to uneven illumination, meets the requirements of multi-shooting angle result discrimination, and is suitable for use in handheld devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0057] Figure 1 A flow chart of a method for determining the results of a multi-directional and multi-scale COVID-19 antigen test provided by an embodiment of the present invention;

[0058] Figure 2 A priori structural elements for construction;

[0059] Figure 3a The original image of the antigen detection result provided in the embodiment of the present invention;

[0060] Figure 3b The result of the priori flexible morphological filtering provided in the embodiment of the present invention after removing background interference;

[0061] Figure 3c This is the result after eliminating the influence of reflection interference provided by the embodiment of the present invention;

[0062] Figure 3d The rectangular frame test results of the antigen test kit provided in the embodiment of the present invention;

[0063] Figure 4a The test results of the area to be identified on the test paper provided in the embodiment of the present invention;

[0064] Figure 4b The detection result of the letter-to-be-identified area provided by the embodiment of the present invention;

[0065] Figure 5 A multi-angle training sample of the letter C provided by an embodiment of the present invention;

[0066] Figure 6a The V component C and T letter recognition results provided by the embodiment of the present invention;

[0067] Figure 6b Color histogram adaptive threshold discrimination provided by the embodiment of the present invention;

[0068] Figure 7 The partial original image, stitching result and discrimination result of the embodiment of the present invention are shown. DETAILED DESCRIPTION

[0069] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0070] This embodiment provides a method for automatically distinguishing the results of multi-directional and multi-scale COVID-19 antigen detection. Figure 1 A flowchart of a multi-directional and multi-scale method for determining the results of a COVID-19 antigen test provided by an embodiment of the present invention, see Figure 1 The process specifically includes the following steps:

[0071] Step S1: Perform a priori flexible morphological operator filtering on the input detection image to be judged to eliminate interference from factors such as uneven background illumination.

[0072] Construct a priori structural elements at 0°, 45°, 90°, and 135°, perform a priori flexible morphological operator filtering on the captured antigen test kit test result image to remove noise interference, and perform a priori flexible morphological gradient operation to perform antigen test kit edge detection. The steps include:

[0073] Flexible morphological closing and opening operations are performed in sequence through the combination of 0°, 90° and 45°, 135° structural elements respectively.

[0074]

[0075]

[0076] f is the input image, · represents the closing operation, o represents the opening operation, A1, A2, A3, A4 represent the kernels of the prior structuring elements, B1, B2, B3, B4 are the soft boundaries of the corresponding structuring elements, k is the number of repetitions, g1 is the result of the closing and opening operations of the prior structuring elements A1 and A2, and g2 is the result of the closing and opening operations of the prior structuring elements A3 and A4.

[0077] Figure 2 A priori structural elements for construction; Figure 3a The original image of the antigen detection result provided in the embodiment of the present invention; Figure 3b This is the result of the priori flexible morphological filtering provided by the embodiment of the present invention after removing background interference. In step S1, dilation and erosion operations are performed on g1 and g2 respectively to obtain flexible morphological gradients to extract the edge of the antigen test kit.

[0078]

[0079]

[0080] G=G(g1)+G(g2)

[0081] G(g1) and G(g2) are the morphological gradients of g1 and g2 respectively. is the dilation operation, is the corrosion operation, and G is the edge detection result of the antigen test kit.

[0082] Figure 3c This is the result after eliminating the influence of reflection interference provided by the embodiment of the present invention. Figure 3d The rectangular frame test result of the antigen test kit provided in the embodiment of the present invention. Step S2:

[0083] Through rectangle detection and comprehensive judgment of shape aspect ratio, the area to be judged on the test paper is extracted;

[0084] The number of pixels of the length L and width M of the rectangle is extracted, and the area to be judged of the test paper is extracted by calculating the aspect ratio R. The calculation formula of R is: R=L / M, where the range of R is (5.5, 6.5).

[0085] The rectangle-like index feature is used to describe the degree of approximation between the detection area and the rectangle. The calculation formula of the rectangle-like index Cmtr is: Cmtr = (L*M) / N A

[0086] where N A The number of pixels contained in the potential area of ​​the test paper. The value range of Cmtr is (0.9, 1.1).

[0087] When both the aspect ratio and the rectangular index characteristics are met, it is judged as the test paper discrimination area, and the interference of reflective areas and other shapes is eliminated.

[0088] Figure 4a This is the test result of the area to be identified on the test paper provided by the embodiment of the present invention.

[0089] In step S2, the potential area is located by the slider, and the information entropy value is calculated to perform preliminary detection of the letter discrimination area; the test paper discrimination area is expanded to the left and right by an L*M matrix, and N*M letter discriminations are extracted in this matrix by sliding the slider in sequence. The slider step size St can be adjusted, and the default value St = 0.25L, forming 4 letter discrimination areas (R1, R2, R3, R4).

[0090] Calculate the entropy values ​​of R1, R2, R3, and R4 in sequence, H(R i ), i=1, 2, 3, 4,

[0091]

[0092] Where i = 1, 2, 3, 4, n = N*M, p j is the probability of pixel j appearing;

[0093] The letter discrimination area is Max((H(R1)+H(R2)),(H(R3)+H(R4))). This article takes R3 and R4 as examples.

[0094] Figure 4b This is the detection result of the letter-to-be-identified area provided by the embodiment of the present invention.

[0095] Step S3: Model the C and T letter samples in the angle dimension to form a training set, combine them into a tensor form, and extract the expanded form to extract the sample feature matrix.

[0096] Expand the sample sets of letters C and T to adapt to the recognition of letters C and T under different conditions;

[0097] The C and T letter samples are rotated 360 degrees, and the rotation interval angle is set to a, where a is 15 degrees. The modeling training has a total of 24 angles, generating a sample set, and the sample set has 48 feature vectors;

[0098] The size of each sample is 16*16, and the features are extracted after the samples are expanded, and the feature dimension is 256. The principal components are obtained by PCA transformation, and the first 40 eigenvalues ​​are taken to form a 48*40 sample feature matrix MF.

[0099] Figure 5 This is a multi-angle training sample of the letter C provided by an embodiment of the present invention.

[0100] Step S4: converting the original RGB color image into the HSV color space, extracting features from the letter image blocks, and constructing a feature matching operator to classify the letters C and T. The steps include:

[0101] Step 1: Convert the RGB color image to the HSV color space, use the optimal brightness space V to describe the color change variable, and let the RGB values ​​be Vr, Vg, and Vb respectively;

[0102] When Vr>Vg>Vb or Vr>Vb>Vg, V=Vr;

[0103] When Vb>Vg>Vr or Vb>Vr>Vg, V=Vb;

[0104] When Vg>Vr>Vb or Vg>Vb>Vr, V=Vg;

[0105] Extract the letter image block to be detected from the image of the optimal brightness space V, use PCA transformation to obtain its principal component, take the first 40 eigenvalues, and form a vector with a feature dimension of 40

[0106] Step 2: Construct feature matching operator S is the covariance matrix of MF, is the feature vector of the letter image block to be detected,

[0107] That is the corresponding matching result, the corresponding This is the letter recognition result.

[0108] Figure 6a This is the V component C and T letter recognition result provided by the embodiment of the present invention.

[0109] Step S5: performing color histogram statistics on the extracted test paper discrimination area, adaptively calculating the color segmentation threshold, and detecting the red discrimination mark;

[0110] Red=(R>T1)&(G>T2)&(G<T3)&(B> T4);

[0111] Among them, the values ​​of T1, T2, T3, and T4 are 140, 30, 150, and 20 respectively.

[0112] Figure 6b Color histogram adaptive threshold discrimination provided by the embodiment of the present invention;

[0113] Step S6: The letter discrimination results and the red discrimination mark detection results are spliced ​​to give the negative, positive, and invalid discrimination results of the antigen test.

[0114] Figure 7 For the four types of antigen detection results and discrimination methods provided in the embodiments of the present invention, refer to Figure 7 ;

[0115] In step S6, the test results of the red part of the test paper discrimination area and the test results of the letters C and T are spliced ​​together to form a total of 4 types. The results are judged by the table lookup method, namely type 1 is negative, type 2 is positive, and types 3 and 4 are invalid, thereby giving the judgment results.

[0116] The details are as follows:

[0117] C has result, T has no result, negative;

[0118] C has result, T has result, positive;

[0119] C has no result, T has result, invalid;

[0120] C has results, T has no results, invalid.

[0121] It should be noted that the method for automatically distinguishing the results of the new coronavirus antigen test used in this embodiment can be implemented in the form of FPGA or DSP hardware, or it can be installed and executed in the form of software on computers, mobile phones, etc.

[0122] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A method for automatically distinguishing the results of multi-directional and multi-scale COVID-19 antigen detection, characterized in that: The method comprises the following steps: photographing an original image of an antigen test kit test result, performing a priori flexible morphological operator filtering on the test result image to remove noise interference, and performing a priori flexible morphological gradient operation to extract the edge of the antigen test kit; Through the comprehensive judgment of quasi-rectangle detection and shape aspect ratio, the area to be judged on the test paper is extracted and the interference of the reflective area is eliminated; Use the slider to locate the potential area and calculate the information entropy value to pre-detect the letter discrimination area; Model the C and T letters in the angle dimension to form a training set, combine the samples into a tensor form, and extract the expanded sample feature matrix; Convert the RGB color image to the HSV color space, extract the letter image blocks to be recognized in the optimal brightness space V, expand the letter image blocks to extract features, and construct a feature matching operator to classify the letters C and T; Performing color histogram statistics on the area to be judged on the test paper, adaptively calculating the color segmentation threshold, and detecting the red judgment mark; The letter classification result and the red discrimination mark detection result are spliced ​​to give the result of the antigen test paper test; Construct a priori structural elements at 0°, 45°, 90°, and 135°, perform a priori flexible morphological operator filtering on the captured antigen test kit test result image to remove noise interference, and perform a priori flexible morphological gradient operation to perform antigen test kit edge detection. The steps include: The flexible morphological closing and opening operations are performed sequentially through the combination of 0°, 90° and 45°, 135° structural elements respectively; f is the input image, · represents the closing operation, o represents the opening operation, A1, A2, A3, A4 represent the kernels of the prior structure elements, B1, B2, B3, B4 are the soft boundaries of the corresponding structure elements, k is the number of repetitions, g1 is the result of the closing and opening operations of the prior structure elements A1 and A2, and g2 is the result of the closing and opening operations of the prior structure elements A3 and A4; Perform dilation and erosion operations on g1 and g2 respectively to obtain the flexible morphological gradient: G=G(g1))+G(g2) G(g1) and G(g2) are the morphological gradients of g1 and g2 respectively. is the dilation operation, is the corrosion operation, and G is the edge detection result of the antigen test kit.

2. The method for automatically distinguishing the results of multi-directional and multi-scale COVID-19 antigen detection according to claim 1, characterized in that: The rectangular frame detection is performed by comprehensively judging the shape aspect ratio and the like rectangle detection, and the steps include: Step S1: Extract the number of pixels of the length L and width M of the rectangle, and extract the area to be judged by calculating the aspect ratio R. The calculation formula of R is: R = L / M, where the range of R is (5.5, 6.5); The rectangle-like index feature is used to describe the degree of approximation between the detection area and the rectangle. The calculation formula of the rectangle-like index Cmtr is: Cmtr=(L*M) / N A where N A is the number of pixels contained in the potential area of ​​the test paper, and the value range of Cmtr is (0.9, 1.1); When both the aspect ratio and the rectangular index characteristics are met, it is judged as the test paper discrimination area, and the interference of reflective areas and other shapes is eliminated; Step S2: The test paper discrimination area is expanded to a matrix of L*M on both sides. In this matrix, N*M letter discriminations are extracted in sequence by sliding the slider. The slider step size St can be adjusted. The default value is St = 0.25L, forming four letter discrimination areas (R1, R2, R3, R4); Calculate the entropy values ​​of R1, R2, R3, and R4 in sequence, H(R i ),i=1,2,3,4, Where i = 1, 2, 3, 4, n = N*M, p j is the probability of pixel j appearing; The letter discrimination area is Max((H(R1)+H(R2)),(H(R3)+H(R4))).

3. The method for automatically distinguishing the multi-directional and multi-scale COVID-19 antigen detection results according to claim 1, characterized in that: The method of modeling the C and T letters in the angle dimension to form a training set, then combining the samples into a tensor form, and extracting the expanded sample feature matrix includes: Expand the sample sets of letters C and T to adapt to the recognition of letters C and T under different conditions; The C and T letter samples are rotated 360 degrees, and the rotation interval angle is set to a, where a is 15 degrees. The modeling training has a total of 24 angles, generating a sample set, and the sample set has 48 feature vectors; The size of each sample is 16*16, and the features are extracted after the samples are expanded, and the feature dimension is 256; PCA transformation is used to obtain the principal components, and the first 40 eigenvalues ​​are taken to form a 48*40 sample feature matrix MF.

4. The method for automatically distinguishing multi-directional and multi-scale COVID-19 antigen detection results according to claim 1, characterized in that: The step of classifying the letters C and T further comprises: Step 1: Convert the RGB color image to the HSV color space, use the optimal brightness space V to describe the color change variable, and let the RGB values ​​be Vr, Vg, and Vb respectively; When Vr>Vg>Vb or Vr>Vb>Vg, V=Vr; When Vb>Vg>Vr or Vb>Vr>Vg, V=Vb; When Vg>Vr>Vb or Vg>Vb>Vr, V=Vg; Extract the letter image block to be detected from the image of the optimal brightness space V, use PCA transformation to obtain its principal component, take the first 40 eigenvalues, and form a vector with a feature dimension of 40 Step 2: Construct feature matching operator S is the covariance matrix of MF, is the feature vector of the letter image block to be detected, That is the corresponding matching result, the corresponding This is the letter recognition result.

5. The method for automatically distinguishing multi-directional and multi-scale COVID-19 antigen detection results according to claim 1, characterized in that: The determination method for detecting the red identification mark is as follows: Red=(R>T1)&(G>T2)&(G<T3)&(B> T4); Among them, the values ​​of T1, T2, T3, and T4 are 140, 30, 150, and 20 respectively.

6. The method for automatically distinguishing multi-directional and multi-scale COVID-19 antigen detection results according to claim 1, characterized in that: The result of splicing the letter classification result and the red discrimination mark detection result is judged by the table lookup method. There are four types of results, namely: C has result, T has no result, negative; C has result, T has result, positive; C has no result, T has result, invalid; C has results, T has no results, invalid.

7. An automatic discrimination software for multi-directional and multi-scale COVID-19 antigen detection results, using the automatic discrimination method according to any one of claims 1 to 6, characterized in that: The automatic discrimination software is installed and executed in the form of computer and mobile phone software, and can automatically judge the results of the antigen test based on the original image of the antigen test paper test results and display negative or positive on the computer and mobile phone screens according to the results.

Images