Experimental instrument image recognition and key information extraction method based on artificial intelligence

Abstract

The invention relates to the technical field of image recognition, in particular to an experimental instrument image recognition and key information extraction method based on artificial intelligence. The method comprises the following steps: acquiring an experimental instrument image, and performing principal component analysis and dimension reduction processing on the experimental instrument image to obtain k principal component image features; performing flag bit fusion on the k principal component image features according to preset fusion information to obtain a fused feature vector; calculating a feature similarity value between the fused feature vector and a preset experimental instrument image feature in a feature vector library; the image category corresponding to the experimental instrument image is determined based on the feature similarity value, and the key information of the experimental instrument image is extracted based on the image category, so that the instrument can be quickly and accurately identified and the key information is extracted.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an artificial intelligence-based experimental instrument image recognition and key information extraction method. Background Art

[0002] With the rapid development of science and technology, laboratory instruments are increasingly used in scientific research, teaching, industrial production and other fields. The traditional method of laboratory instrument information management mainly relies on manual recording and sorting, which is not only inefficient but also prone to errors. Especially in large-scale experimental environments, facing massive amounts of laboratory instrument image data, how to quickly and accurately identify instruments and extract key information has become an urgent problem to be solved.

[0003] In the prior art, there is no method that can automatically adapt to images of experimental instruments of different shapes and types, accurately identify the instruments and extract key information, such as the name, model, quantity, location and size of the experimental instruments.

[0004] Therefore, there is an urgent need for experimental instrument image recognition and key information extraction methods based on artificial intelligence to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide an artificial intelligence-based laboratory instrument image recognition and key information extraction method to solve the technical problem that in the prior art, when faced with massive laboratory instrument image data, the instrument cannot be quickly and accurately identified and key information extracted.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An artificial intelligence-based experimental instrument image recognition and key information extraction method, including:

[0008] Acquire the experimental instrument image, perform principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component image features;

[0009] The k principal component image features are fused according to the preset fusion information to obtain a fused feature vector;

[0010] Calculate the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library;

[0011] The image category corresponding to the experimental instrument image is determined based on the feature similarity value, and the key information of the experimental instrument image is extracted based on the image category.

[0012] Furthermore, the principal component analysis and dimensionality reduction processing of the experimental instrument image is performed to obtain k principal component features, which specifically includes the following process:

[0013] Select m experimental instrument image samples of the same category, each sample contains n candidate image features, and arrange the candidate image features of all samples in columns to obtain an m×n matrix, which is recorded as matrix A:

[0014] A= [ a 1 , a 2 , ..., a m ] ;

[0015] Among them, a i Represents a column vector, and the value of i is [ 1, m ] ;

[0016] Calculate the covariance matrix C of all samples:

[0017]

[0018] Among them, A * represents the conjugate transpose of matrix A;

[0019] The eigenvalue decomposition of the covariance matrix C is:

[0020] C=VDV * ;

[0021] Where V = [ v 1 , v 2 , ..., v n] is the unitary quaternion matrix, V * is the conjugate transpose of the matrix V, D is the diagonal matrix composed of the eigenvalues ​​of the covariance matrix C, D = [ λ 1 ,λ 2 , ..., λ n] ,λ i and v i Respectively represent the eigenvalues ​​and corresponding eigenvectors of the covariance matrix C;

[0022] Project the data into the space of feature vectors and use the formula to calculate the dimension value newBV corresponding to each feature vector i :

[0023]

[0024] Among them, BV i The value is the value of the corresponding dimension in the original sample;

[0025] newBV i Sort them in descending order, and select the first k eigenvectors of the sequence as the principal component image features.

[0026] Furthermore, the k principal component image features are fused with their flags according to the preset fusion information to obtain the fused feature vector, which specifically includes the following process:

[0027] After obtaining k principal component image features, the preset fusion information includes the label ID and the representative flag, and a new vector F is formed based on the k principal component image features and the preset fusion information. cz = { F N , ID, flag } , where F N Represents the image features reconstructed from k principal component image features.

[0028] Furthermore, calculating the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library specifically includes the following process:

[0029]

[0030] Where, j = 1, 2...n; dist is the similarity, represents the jth eigenvector value of the fused eigenvector, It represents the jth feature vector value of the preset experimental instrument image feature in the feature vector library, and n is the number of image features to be selected.

[0031] Furthermore, determining the image category corresponding to the experimental instrument image based on the feature similarity value specifically includes the following process:

[0032] Obtain G feature similarity values, construct a rectangular coordinate system with the feature similarity value as the X-axis and the generation time of the feature similarity value as the Y-axis, mark all feature similarity values ​​in the rectangular coordinate system in the form of points, connect adjacent points in the rectangular coordinate system to generate an experimental instrument category curve, draw perpendicular lines from both ends of the experimental instrument category curve to the X-axis to obtain two start and end line segments, and form a closed figure with the experimental instrument category curve, the two start and end line segments and the X-axis, and mark the total area of ​​the closed figure as the representation value of the image category;

[0033] A preset characterization value interval in which the characterization value is located is queried, and an image category corresponding to the experimental instrument image is queried based on the preset characterization value interval, wherein each preset characterization value interval corresponds to an experimental instrument image category.

[0034] Furthermore, extracting key information of experimental instrument images based on image categories specifically includes the following processes:

[0035] Determine the key information to be extracted based on the image category, wherein each image category corresponds to different key information to be extracted, and the key information includes the name, model, quantity, location and size of the experimental instrument;

[0036] Based on the target detection algorithm Faster R-CNN, key information of experimental instrument images is extracted.

[0037] Compared with the existing solutions, the present invention achieves the following beneficial effects:

[0038] The present invention acquires an experimental instrument image, performs principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component image features; performs flag bit fusion on the k principal component image features according to preset fusion information to obtain a fused feature vector; calculates a feature similarity value between the fused feature vector and a preset experimental instrument image feature in a feature vector library; determines an image category corresponding to the experimental instrument image based on the feature similarity value, and extracts key information of the experimental instrument image based on the image category, so as to quickly and accurately identify the instrument and extract key information, improve the efficiency and accuracy of experimental instrument information management, and provide strong support for subsequent instrument use, maintenance and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0040] Figure 1 It is a workflow diagram of an artificial intelligence-based experimental instrument image recognition and key information extraction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, well-known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0043] This embodiment provides an experimental instrument image recognition and key information extraction method based on artificial intelligence. Figure 1 FIG. 1 is a flowchart of an artificial intelligence-based experimental instrument image recognition and key information extraction method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0044] Step S101: Acquire an experimental instrument image, and perform principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component image features;

[0045] Step S102: fusing the k principal component image features according to the preset fusion information to obtain a fused feature vector;

[0046] Step S103: calculating the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library;

[0047] Step S104: determining the image category corresponding to the experimental instrument image based on the feature similarity value, and extracting key information of the experimental instrument image based on the image category.

[0048] In summary, the present invention acquires an experimental instrument image, performs principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component image features; performs flag fusion on the k principal component image features according to preset fusion information to obtain a fused feature vector; calculates feature similarity values ​​between the fused feature vector and preset experimental instrument image features in a feature vector library; determines the image category corresponding to the experimental instrument image based on the feature similarity values, and extracts key information of the experimental instrument image based on the image category, which can quickly and accurately identify the instrument and extract key information, improve the efficiency and accuracy of experimental instrument information management, and provide strong support for subsequent instrument use, maintenance and management.

[0049] In some embodiments, performing principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component features specifically includes the following process:

[0050] Select m experimental instrument image samples of the same category, each sample contains n candidate image features, and arrange the candidate image features of all samples in columns to obtain an m×n matrix, which is recorded as matrix A:

[0051] A= [ a 1 , a 2 , ..., a m ] ;

[0052] Among them, a i Represents a column vector, and the value of i is [ 1, m ] ;

[0053] Calculate the covariance matrix C of all samples:

[0054]

[0055] Among them, A * represents the conjugate transpose of matrix A;

[0056] The eigenvalue decomposition of the covariance matrix C is:

[0057] C=VDV * ;

[0058] Where V = [ v 1 , v 2 , ..., v n] is the unitary quaternion matrix, V * is the conjugate transpose of the matrix V, D is the diagonal matrix composed of the eigenvalues ​​of the covariance matrix C, D = [ λ 1 ,λ 2 , ..., λ n] ,λ i and v i Respectively represent the eigenvalues ​​and corresponding eigenvectors of the covariance matrix C;

[0059] Project the data into the space of feature vectors and use the formula to calculate the dimension value newBV corresponding to each feature vector i :

[0060]

[0061] Among them, BV i The value is the value of the corresponding dimension in the original sample;

[0062] newBV i Sort them in descending order, and select the first k eigenvectors of the sequence as the principal component image features.

[0063] In some embodiments, fusing the k principal component image features with their flags according to preset fusion information to obtain a fused feature vector specifically includes the following process:

[0064] After obtaining k principal component image features, the preset fusion information includes the label ID and the representative flag, and a new vector F is formed based on the k principal component image features and the preset fusion information. cz = { F N , ID, flag } , where F N Represents the image features reconstructed from k principal component image features.

[0065] In some embodiments, calculating the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library specifically includes the following process:

[0066]

[0067] Where, j = 1, 2...n; dist is the similarity, represents the jth eigenvector value of the fused eigenvector, It represents the jth feature vector value of the preset experimental instrument image feature in the feature vector library, and n is the number of image features to be selected.

[0068] In some embodiments, determining the image category corresponding to the experimental instrument image based on the feature similarity value specifically includes the following process:

[0069] Obtain G feature similarity values, construct a rectangular coordinate system with the feature similarity value as the X-axis and the generation time of the feature similarity value as the Y-axis, mark all feature similarity values ​​in the rectangular coordinate system in the form of points, connect adjacent points in the rectangular coordinate system to generate an experimental instrument category curve, draw perpendicular lines from both ends of the experimental instrument category curve to the X-axis to obtain two start and end line segments, and form a closed figure with the experimental instrument category curve, the two start and end line segments and the X-axis, and mark the total area of ​​the closed figure as the representation value of the image category;

[0070] A preset characterization value interval in which the characterization value is located is queried, and an image category corresponding to the experimental instrument image is queried based on the preset characterization value interval, wherein each preset characterization value interval corresponds to an experimental instrument image category.

[0071] In some embodiments, extracting key information of an experimental instrument image based on image category specifically includes the following process:

[0072] Determine the key information to be extracted based on the image category, wherein each image category corresponds to different key information to be extracted, and the key information includes the name, model, quantity, location and size of the experimental instrument;

[0073] Based on the target detection algorithm Faster R-CNN, key information of experimental instrument images is extracted.

[0074] Specifically, data preparation:

[0075] Collect image datasets containing experimental instruments and annotate them. The annotation information includes the name, model, quantity, location, size, etc. of the experimental instruments.

[0076] Divide the dataset into training, validation, and test sets.

[0077] Model training:

[0078] Choose the appropriate deep learning framework target detection algorithm Faster R-CNN;

[0079] Construct a network model and modify and optimize it appropriately according to the characteristics of the experimental instrument images.

[0080] Use the training set to train the model, and through continuous modification and calculation of the optimizer and loss function, gradually adjust the model parameters to improve the recognition accuracy of the model.

[0081] Model Validation and Testing:

[0082] Use the validation set to validate the trained model and evaluate the performance of the model.

[0083] Adjust and optimize the model based on the verification results.

[0084] The final model is tested using the test set to ensure the accuracy and stability of the model.

[0085] Key information extraction:

[0086] The trained model is used to perform target detection on the experimental instrument images to automatically detect the name, model, quantity, location, size and other information of the experimental instruments.

[0087] Extract key information such as name, model, quantity, location and size based on the detection results.

[0088] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0089] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0091] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0092] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0093] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An artificial intelligence-based experimental instrument image recognition and key information extraction method, characterized in that the method include: Acquire the experimental instrument image, perform principal component analysis and dimensionality reduction processing on the experimental instrument image to obtain k principal component image features; The k principal component image features are fused according to the preset fusion information to obtain a fused feature vector; Calculate the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library; The image category corresponding to the experimental instrument image is determined based on the feature similarity value, and the key information of the experimental instrument image is extracted based on the image category.

2. The method for experimental instrument image recognition and key information extraction based on artificial intelligence according to claim 1, characterized in that: The principal component analysis and dimensionality reduction processing of the experimental instrument image is performed to obtain k principal component features. The process includes: Select m experimental instrument image samples of the same category, each sample contains n candidate image features, and arrange the candidate image features of all samples in columns to obtain an m×n matrix, which is recorded as matrix A: <h2 style=";text-align:left;direction:ltr">A=<h2 style=";text-align:left;direction:ltr"> [ <h2 style=";text-align:left;direction:ltr"> a1, a2,..., a<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr"> ] <h2 style=";text-align:left;direction:ltr"> ; Among them, a i Represents a column vector, and the value of i is [ 1, m ] ; Calculate the covariance matrix C of all samples: Among them, A * represents the conjugate transpose of matrix A; The eigenvalue decomposition of the covariance matrix C is: C=VDV * ; Where V = [ v1, v2, ..., v n] is the unitary quaternion matrix, V * is the conjugate transpose of the matrix V, D is the diagonal matrix composed of the eigenvalues ​​of the covariance matrix C, D = [ λ1, λ2, ..., λ n] ,λ i and v i Respectively represent the eigenvalues ​​and corresponding eigenvectors of the covariance matrix C; Project the data into the space of feature vectors and use the formula to calculate the dimension value newBV corresponding to each feature vector i : Among them, BV i The value is the value of the corresponding dimension in the original sample; newBV i Sort them in descending order, and select the first k eigenvectors of the sequence as the principal component image features.

3. The method for experimental instrument image recognition and key information extraction based on artificial intelligence according to claim 2, characterized in that: The k principal component image features are fused with the flags according to the preset fusion information to obtain the fused feature vector, which specifically includes the following process: After obtaining k principal component image features, the preset fusion information includes the label ID and the representative flag, and a new vector F is formed based on the k principal component image features and the preset fusion information. cz = { F N , ID, flag } , where F N Represents the image features reconstructed from k principal component image features.

4. The method for experimental instrument image recognition and key information extraction based on artificial intelligence according to claim 1, characterized in that: The specific process of calculating the feature similarity value between the fused feature vector and the preset experimental instrument image features in the feature vector library includes the following steps: Where, j = 1, 2...n; dist is the similarity, represents the jth eigenvector value of the fused eigenvector, It represents the jth feature vector value of the preset experimental instrument image feature in the feature vector library, and n is the number of image features to be selected.

5. The method for experimental instrument image recognition and key information extraction based on artificial intelligence according to claim 4 is characterized in that: Determining the image category corresponding to the experimental instrument image based on the feature similarity value specifically includes the following process: Obtain G feature similarity values, construct a rectangular coordinate system with the feature similarity value as the X-axis and the generation time of the feature similarity value as the Y-axis, mark all feature similarity values ​​in the rectangular coordinate system in the form of points, connect adjacent points in the rectangular coordinate system to generate an experimental instrument category curve, draw perpendicular lines from both ends of the experimental instrument category curve to the X-axis to obtain two start and end line segments, and form a closed figure with the experimental instrument category curve, the two start and end line segments and the X-axis, and mark the total area of ​​the closed figure as the representation value of the image category; A preset characterization value interval in which the characterization value is located is queried, and an image category corresponding to the experimental instrument image is queried based on the preset characterization value interval, wherein each preset characterization value interval corresponds to an experimental instrument image category.

6. The method for experimental instrument image recognition and key information extraction based on artificial intelligence according to claim 1, characterized in that: Extracting key information of experimental instrument images based on image categories specifically includes the following processes: Determine the key information to be extracted based on the image category, wherein each image category corresponds to different key information to be extracted, and the key information includes the name, model, quantity, location and size of the experimental instrument; Based on the target detection algorithm Faster R-CNN, key information of experimental instrument images is extracted.

Images