Power enterprise e-commerce material information automatic extraction and verification system and method based on CV large model and OCR

Abstract

The invention discloses an electric power enterprise e-commerce material information automatic extraction and verification system and method based on a CV large model and OCR identification. The system comprises an image acquisition module, an image analysis module, a comparison and verification module and a server. The image acquisition module comprises an industrial camera and is used for acquiring image data of materials; the image analysis module is internally provided with a CV large model and an OCR recognition module, and the CV large model is used for carrying out image analysis on the collected image data and recognizing the types and specifications of the materials; the OCR recognition module is used for recognizing character information of material labels in the image data; the comparison and verification module is used for comparing the material information of the identified image data with standard information in a database, and verifying whether material distribution is correct or not; the problems that the efficiency is low and the accuracy is poor due to the fact that the existing power enterprise material acceptance and warehousing process depends on manual information input and verification are solved. And the operation efficiency and the accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of goods acceptance of electric power enterprises, and in particular to a system and method for automatically extracting and verifying e-commerce material information of electric power enterprises based on a CV large model and OCR recognition. Background Art

[0002] At present, power companies need to purchase a large amount of materials in their production operations, including online and offline purchases, which generates a large amount of logistics activities for material transportation. Due to the inconsistent management levels and operating methods of different downstream suppliers, there is a lack of unified management standards when receiving and accepting materials from multiple suppliers. The existing technology generally relies on a large number of manual checks and acceptances one by one, which is not only inefficient, but also consumes a lot of manpower. When faced with a large amount of goods entering the warehouse in a short period of time, it is easy to have acceptance omissions. Manual information registration and comparison of incoming materials are prone to errors, resulting in material information entry errors or comparison errors, affecting subsequent use. Therefore, it is necessary to design an automatic extraction and verification system and method for e-commerce material information of power companies based on CV large models and OCR recognition to solve the above problems. Summary of the invention

[0003] The purpose of the present invention is to provide a system and method for automatic extraction and verification of e-commerce material information of electric power enterprises based on CV big model and OCR recognition, aiming to solve the problem that the existing material acceptance and warehousing process of electric power enterprises relies on manual information entry and verification, which is inefficient and has poor accuracy; it has the characteristics of automatically extracting and comparing material information through CV big model, greatly reducing the degree of manual intervention, thereby improving work efficiency and accuracy.

[0004] In order to achieve the above technical effects, the technical solution adopted by the present invention is: An automatic extraction and verification system for e-commerce material information of electric power enterprises based on CV large model and OCR recognition, including image acquisition module, image analysis module, comparison and verification module and server; The image acquisition module includes an industrial camera for collecting image data of materials; The image analysis module has a built-in CV large model and OCR recognition module. The CV large model is used to analyze the collected image data and identify the types and specifications of materials; the OCR recognition module is used to identify the text information of the material labels in the image data; The comparison and verification module is used to compare the material information of the recognized image data with the standard information in the database to verify whether the material distribution is correct; The server has multiple built-in databases for storing collected image data, material information obtained from image analysis, and standard information stored in the database.

[0005] The image acquisition module, image analysis module and comparison and verification module are all connected to the server through the communication module.

[0006] Preferably, the communication module includes a wireless connection and a wired connection.

[0007] Preferably, the standard information includes the type, specification, manufacturer and model of the material.

[0008] Preferably, the text information on the material label includes the type, specification, manufacturer and model of the material.

[0009] Preferably, the above-mentioned method of the automatic extraction and verification system of the electric power enterprise e-commerce material information based on the CV large model and OCR recognition comprises the following steps: S1, uses industrial cameras to capture multi-angle images of current materials and label information; S2, based on the CV large model, performs image analysis on multi-angle images to identify the types and specifications of materials in the image data; S3, recognizing text information in the label information based on the OCR recognition module; S4, integrates the image analysis results of the CV large model and the OCR recognition module to construct the comprehensive data information of the current materials in the form of an array; S5, compare the comprehensive data information of the current materials with the standard information in the database. When the data are consistent, it is determined to be qualified. If not, it is determined as a material acceptance error.

[0010] Preferably, the CV large model is constructed using an improved ResNet model, and the construction method is: S201, create an image training set: Collect multiple photos of materials from the historical database of received materials as training sets. The original images of the material photos are RGB images of r*c size. S202, training set data preprocessing: Extract whitening features and LBP features from the original image; S203, principal component analysis replacement processing: Read the original RGB image to obtain the corresponding matrix T0, T0 is an r*c*3 matrix; read the whitened image matrix M0 as r*c, read the LBP matrix as r*c, recorded as F0; Perform matrix data normalization preprocessing: T=T0 / 255; M=M0 / 255; F=F0 / 255; Transform the matrix of T into a (r*c)*3 matrix: Transform each channel matrix of T into a row, denoted as Ts, and then find the correlation matrix of T, denoted as T1: T1 = (T'*T) / (r*c); Where T' is the transposed matrix of T; Find the eigenvalue value and eigenvector vector to get the principal component matrix pa: pa = Ts*vector; The principal component matrix pa is transformed into a matrix pas of r*c*3; based on the eigenvalue value, the third channel of pas is the first principal component. The first principal component is retained, and the remaining two channels are normalized, that is, the elements of each channel are between 0 and 1. Then, the whitened features and LBP features are matched with the remaining two smaller channel principal components by histogram matching, that is, the histograms of M and F are matched with the histograms of the second and third principal components. Figure 1 To; After the histogram matching is completed, M and F replace the matrices of the second and third principal components respectively, and then the fused image is reconstructed through the inverse PCA transform to form a new RGB image; S204, improved ResNet: Keep the convolution of the model unchanged, remove the last average pooling layer and fully connected layer of the model to obtain r feature maps, and then add Relu layer and convolution layer in sequence, where the convolution kernel size is 3*3 and the step size is 2. The convolution layer obtains r feature maps after downsampling, and then adds a batch normalization layer; the average pooling layer obtains r 1*1 feature maps, which are then transformed into r*1 vectors, and finally pass through the fully connected layer to obtain the classification result; S205, training the improved ResNet model: constructing a training sample set through the new RGB image generated in step S203, training the improved ResNet model, and obtaining a trained CV large model.

[0011] Preferably, the method for recognizing text information in the label information based on the OCR recognition module is: S301, segmenting the label image in the collected image data based on the CV large model to segment the label or nameplate image; S302, performing image segmentation and extraction on the white background and black lines in the label image respectively, and uniformly numbering the white background and black lines in the same group; S303, performing denoising processing on all extracted white backgrounds and black lines respectively; S304, merging the denoised white background and black lines to form an image to be recognized; S305, matching the image to be recognized with the preset coordinate template, after matching, obtaining a plurality of groups of line segments to be processed from the recognized image, from the starting point and the end point inside the line segments to be processed, marking the starting point as QSi, marking the end point as MDi, where i represents different line segments to be processed; Get the coordinates (X1, Y1) of the starting point QSi, and get the coordinates (X2, Y2) of the end point MDi; calculate the line equation Y=KX+H between the starting point QSi and the end point MDi, and the calculation method is: ; Substitute multiple groups of points inside the line segment into the line equation Y=KX+H to check whether the coordinate parameters of the point values ​​meet the line equation Y=KX+H. If so, a normal signal is generated; if not, a defect signal is generated. Multiple groups of different line segments are processed according to the above steps to obtain corresponding normal signals and defect signals. When there is no defect signal inside the image to be identified, the high-definition image is directly output through the output terminal.

[0012] Preferably, when the material type and specification identified by the CV large model are inconsistent with the material type and specification identified by the label, the parameters of the CV large model are calibrated.

[0013] Preferably, during the data processing, the material information and the acceptance results are encrypted by encryption technology.

[0014] The beneficial effects of the system and method for automatically extracting and verifying e-commerce material information of electric power enterprises based on CV large model and OCR recognition provided by the present invention are as follows: This application solves the problem that the existing power company material acceptance and warehousing process relies on manual information entry and verification, which is inefficient and inaccurate; it has the characteristics of automatically extracting and comparing material information through a CV large model, greatly reducing the degree of manual intervention, thereby improving operational efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] Embodiment 1: like Figure 1 As shown, a system for automatically extracting and verifying e-commerce material information of electric power enterprises based on CV large model and OCR recognition includes an image acquisition module, an image analysis module, a comparison and verification module and a server; The image acquisition module includes an industrial camera for collecting image data of materials; The image analysis module has a built-in CV large model and OCR recognition module. The CV large model is used to analyze the collected image data and identify the types and specifications of materials; the OCR recognition module is used to identify the text information of the material labels in the image data; The comparison and verification module is used to compare the material information of the recognized image data with the standard information in the database to verify whether the material distribution is correct; The server has multiple built-in databases for storing collected image data, material information obtained from image analysis, and standard information stored in the database.

[0017] The image acquisition module, image analysis module and comparison and verification module are all connected to the server through the communication module.

[0018] Preferably, the communication module includes a wireless connection and a wired connection.

[0019] Preferably, the standard information includes the type, specification, manufacturer and model of the material.

[0020] Preferably, the text information on the material label includes the type, specification, manufacturer and model of the material.

[0021] Embodiment 2: like Figure 2 As shown, the above-mentioned method of the automatic extraction and verification system of the e-commerce material information of the power enterprise based on the CV large model and OCR recognition includes the following steps: S1, uses industrial cameras to capture multi-angle images of current materials and label information; S2, based on the CV large model, performs image analysis on multi-angle images to identify the types and specifications of materials in the image data; S3, recognizing text information in the label information based on the OCR recognition module; S4, integrates the image analysis results of the CV large model and the OCR recognition module to construct the comprehensive data information of the current materials in the form of an array; S5, compare the comprehensive data information of the current materials with the standard information in the database. When the data are consistent, it is determined to be qualified. If not, it is determined as a material acceptance error.

[0022] Preferably, the CV large model is constructed using an improved ResNet model, and the construction method is: S201, create an image training set: Collect multiple photos of materials from the historical database of received materials as training sets. The original images of the material photos are RGB images of r*c size. S202, training set data preprocessing: Extract whitening features and LBP features from the original image; S203, principal component analysis replacement processing: Read the original RGB image to obtain the corresponding matrix T0, T0 is an r*c*3 matrix; read the whitened image matrix M0 as r*c, read the LBP matrix as r*c, recorded as F0; Perform matrix data normalization preprocessing: T=T0 / 255; M=M0 / 255; F=F0 / 255; Transform the matrix of T into a (r*c)*3 matrix: Transform each channel matrix of T into a row, denoted as Ts, and then find the correlation matrix of T, denoted as T1: T1 = (T'*T) / (r*c); Where T' is the transposed matrix of T; Find the eigenvalue value and eigenvector vector to get the principal component matrix pa: pa = Ts*vector; The principal component matrix pa is transformed into a matrix pas of r*c*3; based on the eigenvalue value, the third channel of pas is the first principal component. The first principal component is retained, and the remaining two channels are normalized, that is, the elements of each channel are between 0 and 1. Then, the whitened features and LBP features are matched with the remaining two smaller channel principal components by histogram matching, that is, the histograms of M and F are matched with the histograms of the second and third principal components. Figure 1 To; After the histogram matching is completed, M and F replace the matrices of the second and third principal components respectively, and then the fused image is reconstructed through the inverse PCA transform to form a new RGB image; S204, improved ResNet: Keep the convolution of the model unchanged, remove the last average pooling layer and fully connected layer of the model to obtain r feature maps, and then add Relu layer and convolution layer in sequence, where the convolution kernel size is 3*3 and the step size is 2. The convolution layer obtains r feature maps after downsampling, and then adds a batch normalization layer; the average pooling layer obtains r 1*1 feature maps, which are then transformed into r*1 vectors, and finally pass through the fully connected layer to obtain the classification result; S205, training the improved ResNet model: constructing a training sample set through the new RGB image generated in step S203, training the improved ResNet model, and obtaining a trained CV large model.

[0023] Preferably, the method for recognizing text information in the label information based on the OCR recognition module is: S301, segmenting the label image in the collected image data based on the CV large model to segment the label or nameplate image; S302, performing image segmentation and extraction on the white background and black lines in the label image respectively, and uniformly numbering the white background and black lines in the same group; S303, performing denoising processing on all extracted white backgrounds and black lines respectively; S304, merging the denoised white background and black lines to form an image to be recognized; S305, matching the image to be recognized with the preset coordinate template, after matching, obtaining a plurality of groups of line segments to be processed from the recognized image, from the starting point and the end point inside the line segments to be processed, marking the starting point as QSi, marking the end point as MDi, where i represents different line segments to be processed; Get the coordinates (X1, Y1) of the starting point QSi, and get the coordinates (X2, Y2) of the end point MDi; calculate the line equation Y=KX+H between the starting point QSi and the end point MDi, and the calculation method is: ; Substitute multiple groups of points inside the line segment into the line equation Y=KX+H to check whether the coordinate parameters of the point values ​​meet the line equation Y=KX+H. If so, a normal signal is generated; if not, a defect signal is generated. Multiple groups of different line segments are processed according to the above steps to obtain corresponding normal signals and defect signals. When there is no defect signal inside the image to be identified, the high-definition image is directly output through the output terminal.

[0024] Preferably, when the material type and specification identified by the CV large model are inconsistent with the material type and specification identified by the label, the parameters of the CV large model are calibrated.

[0025] Preferably, during the data processing, the material information and the acceptance results are encrypted by encryption technology.

Claims

1. An automatic extraction and verification system for e-commerce material information of power enterprises based on CV large model and OCR recognition, characterized by: It includes an image acquisition module, an image analysis module, a comparison and verification module and a server; The image acquisition module includes an industrial camera for collecting image data of materials; The image analysis module has a built-in CV large model and OCR recognition module. The CV large model is used to analyze the collected image data and identify the types and specifications of materials; the OCR recognition module is used to identify the text information of the material labels in the image data; The comparison and verification module is used to compare the material information of the recognized image data with the standard information in the database to verify whether the material distribution is correct; The server has multiple built-in databases for storing collected image data, material information obtained from image analysis, and standard information stored in the database; The image acquisition module, image analysis module and comparison and verification module are all connected to the server through the communication module.

2. According to claim 1, a system for automatically extracting and verifying e-commerce material information of electric power enterprises based on CV large model and OCR recognition is characterized by: The communication module includes wireless connection and wired connection.

3. According to claim 1, a system for automatically extracting and verifying e-commerce material information of electric power enterprises based on CV large model and OCR recognition is characterized by: Standard information includes the type, specification, manufacturer and model of materials.

4. According to claim 1, a system for automatically extracting and verifying e-commerce material information of electric power enterprises based on CV large model and OCR recognition is characterized by: The text information on the material label includes the type, specification, manufacturer and model of the material.

5. A method for automatic extraction and verification of e-commerce material information of electric power enterprises based on CV large model and OCR recognition according to any one of claims 1 to 4, characterized in that: The following steps are involved: S1, uses industrial cameras to capture multi-angle images of current materials and label information; S2, based on the CV large model, performs image analysis on multi-angle images to identify the types and specifications of materials in the image data; S3, recognizing text information in the label information based on the OCR recognition module; S4, integrates the image analysis results of the CV large model and the OCR recognition module to construct the comprehensive data information of the current materials in the form of an array; S5, compare the comprehensive data information of the current materials with the standard information in the database. When the data are consistent, it is determined to be qualified. If not, it is determined as a material acceptance error.

6. According to claim 5, a method for automatic extraction and verification of electric power enterprise e-commerce material information based on CV large model and OCR recognition is characterized in that: The CV large model is constructed using the improved ResNet model. The construction method is: S201, create an image training set: Collect multiple photos of materials from the historical database of received materials as training sets. The original images of the material photos are RGB images of r*c size. S202, training set data preprocessing: Extract whitening features and LBP features from the original image; S203, principal component analysis replacement processing: Read the original RGB image to obtain the corresponding matrix T0, T0 is an r*c*3 matrix; read the whitened image matrix M0 as r*c, read the LBP matrix as r*c, recorded as F0; Perform matrix data normalization preprocessing: T=T0 / 255; M=M0 / 255; F=F0 / 255; Transform the matrix of T into a (r*c)*3 matrix: Transform each channel matrix of T into a row, denoted as Ts, and then find the correlation matrix of T, denoted as T1: T1 = (T'*T) / (r*c); Where T' is the transposed matrix of T; Find the eigenvalue value and eigenvector vector to get the principal component matrix pa: pa = Ts*vector; The principal component matrix pa is transformed into a matrix pas of r*c*3; based on the eigenvalue value, the third channel of pas is the first principal component. The first principal component is retained, and the remaining two channels are normalized, that is, the elements of each channel are between 0 and 1. Then, the whitened features and LBP features are respectively matched with the remaining two smaller channel principal components by histogram matching, that is, the histograms of M and F are consistent with the histograms of the second and third principal components; After the histogram matching is completed, M and F replace the matrices of the second and third principal components respectively, and then the fused image is reconstructed through the inverse PCA transform to form a new RGB image; S204, improved ResNet: Keep the convolution of the model unchanged, remove the last average pooling layer and fully connected layer of the model to obtain r feature maps, and then add Relu layer and convolution layer in sequence, where the convolution kernel size is 3*3 and the step size is 2. The convolution layer obtains r feature maps after downsampling, and then adds a batch normalization layer; the average pooling layer obtains r 1*1 feature maps, which are then transformed into r*1 vectors, and finally pass through the fully connected layer to obtain the classification result; S205, training the improved ResNet model: constructing a training sample set through the new RGB image generated in step S203, training the improved ResNet model, and obtaining a trained CV large model.

7. The method of the automatic extraction and verification system of the e-commerce material information of the power enterprise based on the CV large model and OCR recognition according to claim 6 is characterized in that: The method for recognizing text information in label information based on the OCR recognition module is as follows: S301, segmenting the label image in the collected image data based on the CV large model to segment the label or nameplate image; S302, performing image segmentation and extraction on the white background and black lines in the label image respectively, and uniformly numbering the white background and black lines in the same group; S303, performing denoising processing on all extracted white backgrounds and black lines respectively; S304, merging the denoised white background and black lines to form an image to be recognized; S305, matching the image to be recognized with the preset coordinate template, after matching, obtaining a plurality of groups of line segments to be processed from the recognized image, from the starting point and the end point inside the line segments to be processed, marking the starting point as QSi, marking the end point as MDi, where i represents different line segments to be processed; Get the coordinates (X1, Y1) of the starting point QSi, and get the coordinates (X2, Y2) of the end point MDi; calculate the line equation Y=KX+H between the starting point QSi and the end point MDi, and the calculation method is: ； Substitute multiple groups of points inside the line segment into the line equation Y=KX+H to check whether the coordinate parameters of the point values ​​meet the line equation Y=KX+H. If so, a normal signal is generated; if not, a defect signal is generated. Multiple groups of different line segments are processed according to the above steps to obtain corresponding normal signals and defect signals. When there is no defect signal inside the image to be identified, the high-definition image is directly output through the output terminal.

8. The method of the automatic extraction and verification system of the e-commerce material information of the power enterprise based on the CV large model and OCR recognition according to claim 7 is characterized in that: When the types and specifications of materials recognized by the CV large model are inconsistent with those recognized by the tags, the parameters of the CV large model are calibrated.

9. The method of the automatic extraction and verification system of the e-commerce material information of the power enterprise based on the CV large model and OCR recognition according to claim 8 is characterized in that: During the data processing process, material information and acceptance results are encrypted using encryption technology.

Images