Electric power enterprise e-commerce material automatic acceptance device and method based on image and data fusion

Abstract

The invention discloses a power enterprise e-commerce material automatic acceptance device and method based on image and data fusion. The device comprises a data acquisition module, a weighing measurement module, a logistics tracking module, a classification acceptance module and a server. The data acquisition module comprises an industrial camera and a card reader, and the weighing measurement module comprises a high-precision weighing sensor and is used for acquiring weight data of warehouse-in materials; the logistics tracking module comprises an RFID electronic tag arranged on an outer package of a material and a positioning module, the classification acceptance module comprises an image recognition module and an acceptance module, and the image recognition module executes an image recognition algorithm to perform large-class recognition on the material; the data acquisition module, the weighing measurement module, the logistics tracking module and the classification acceptance module are all in communication connection with the server through the communication module; according to the invention, the problem of high acceptance cost caused by time and labor consumption and poor accuracy due to manual checking and classification in the prior art is solved, and the efficiency and accuracy of material acceptance are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of goods transportation for electric power enterprises, and in particular to an automatic acceptance device and method for e-commerce materials for electric power enterprises based on image and data fusion. Background Art

[0002] Currently, power companies need to purchase a large amount of materials in their production operations, including online and offline purchases, which generates a large amount of logistics activities for material transportation. Due to the inconsistent management levels and operating methods of different downstream suppliers, there is a lack of unified management standards when accepting the receipt of materials from multiple suppliers. A lot of manpower is needed to check them one by one, which is not only inefficient, but also consumes a lot of manpower. When a large amount of goods are put into storage in a short time, it is easy to have omissions in acceptance. Therefore, it is necessary to design an automatic acceptance device and method for e-commerce materials of power companies based on image and data fusion to solve the above problems. Summary of the invention

[0003] The purpose of the present invention is to provide an automatic acceptance device and method for e-commerce materials of electric power enterprises based on image and data fusion, aiming to solve the problem that the existing power enterprise material acceptance and warehousing process relies heavily on manual verification and classification, which is time-consuming and labor-intensive and has poor accuracy, resulting in high acceptance costs; it has the characteristics of being able to automatically perform acceptance through image recognition, classify unqualified materials by type, greatly reduce the degree of manual intervention, and thus improve efficiency and accuracy.

[0004] In order to achieve the above technical effects, the technical solution adopted by the present invention is: An automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion, comprising a data acquisition module, a weighing and measuring module, a logistics tracking module, a classification acceptance module and a server; The data acquisition module includes an industrial camera and a card reader. The industrial camera is used to collect multi-angle images of materials, and the card reader is used to collect logistics information of the logistics tracking module. The weighing measurement module includes a high-precision weighing sensor, which is used to collect the weight data of the materials in the warehouse; The logistics tracking module includes an RFID electronic tag set on the outer packaging of the material and a positioning module. The positioning module is used to collect the location information of the material and the corresponding timestamp to form a "location-time" electronic tag and cooperate with the card reader of the data acquisition module through the RFID electronic tag; The classification and acceptance module includes an image recognition module and an acceptance module. The image recognition module executes an image recognition algorithm to identify the broad categories of materials. The acceptance module subdivides and accepts the materials based on the results of broad category identification. The data acquisition module, weighing measurement module, logistics tracking module and classification acceptance module are all connected to the server through the communication module.

[0005] Preferably, the communication module includes an LTE communication module, a wifi module and a Bluetooth transceiver module.

[0006] Preferably, an image database, a comprehensive acquisition database and a filing database are provided in the server. The server stores image data collected by industrial cameras, the comprehensive acquisition database stores location data, time data and weighing data of materials, and the filing database is communicated with the logistics reporting system for receiving initial data of material reporting.

[0007] Preferably, the logistics reporting system is used to fill in the initial data of the transported materials before the materials are transported. The initial data includes multi-angle images of the uploaded materials, cargo weight information, and logistics distribution shipping location and delivery time.

[0008] Preferably, the image recognition algorithm is used to calculate the weighted histogram similarity DH of the collected cargo images to be identified, and match them with the images in the filing database, so as to identify the major categories of materials.

[0009] Preferably, the above-mentioned method of the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion comprises the following steps: S1, using an industrial camera to capture multi-angle images of materials, by moving the industrial camera to multiple shooting positions or using multiple industrial cameras set at multiple positions to perform multi-angle shooting; S2, record shooting position and histogram information His p , is the azimuth, is the pitch angle; the colleague who took the photo reads the "position-time" data of the RFID electronic tag with a card reader, and the weighing sensor collects the weight information of the current material, stores the above-mentioned multiple data in the form of a string, and records the data for each material in turn; S3, read the initial data submitted in the logistics reporting system and set the image comparison priority in the filing database: The priority is the place of shipment and weight. The data in the record database with the same place of shipment as the current place of shipment of the material to be identified is selected as the data for priority comparison and forms a data set Q. Then, the weight data in the data set Q is subtracted from the weighing data of the current material to be identified, and the multiple comparison materials in the data set Q are sorted in ascending order of the difference value. S3, the image recognition module determines the shooting direction , extract the histogram information HIS of the image features of M adjacent preset positions of the reported comparison materials in the data set Q according to the sorting order J ; S4, image recognition module calculates His p and HIS J Similarity DH j , and further solve the weighted histogram similarity DH of the current material to be identified: ; In the formula, j represents the number of the current position, S represents the area budget, represents the total geometric area of ​​the image in M ​​directions, represents the total geometric area of ​​the image after the current j-th orientation is replaced by orientation V; S5, executing steps S3 to S4 in order for the comparison materials in the data set Q, and calculating the DH values ​​of all the comparison materials; S6, comparing the calculated DH with the set threshold △DH, performing feature matching on the compared materials whose DH is greater than the set threshold △DH, and eliminating the remaining compared materials; For the comparison materials whose feature matching value TZ is greater than the set threshold △TZ, they are identified as candidate targets, and the comparison materials with the largest feature matching value TZ among multiple candidate targets are selected as target comparison materials; TZ is the ratio of feature matching data to the total number of features, TZ satisfies 0≤TZ≤1, and △TZ satisfies 0≤△TZ≤1; S7, comparing the weight information, place of shipment and delivery time reported by the comparison material selected in step S6 with the currently measured material, if all three items are the same, matching the comparison material with the current material to be identified, marking it as "accepted" and entering the warehouse according to the reported information; if the weight information is different, it is determined to be "goods quantity defect" and classified; if the delivery time is different, it is determined to be "material overdue" and classified; S8, in step S6, if there is no material to be compared whose DH is greater than the set threshold △DH, the material to be identified is classified as "abnormal" and manual comparison is performed after all materials are inspected and accepted.

[0010] Preferably, during the data processing, the material information and the acceptance results are encrypted by encryption technology.

[0011] Preferably, in step S3, the method for acquiring the multi-angle images of the materials in the filing database is: S301, from N preset positions Take images of delivered materials i And the corresponding histogram information His i , represented by His i[V0,V1,......,V255],His i is a one-dimensional array with 256 elements, V i Represents the ratio of pixels with gray value i in the image to the total pixels in the image; i is the number indicating the orientation. For preset position The azimuth of For preset position The pitch angle; S302, extracting image T i The feature information FEAT is expressed as FEAT{kp1,kp2,......,kpn}, where kp represents a key point in the image feature information, expressed as kp{σ,Loc,Hes}, σ represents the scale of the scale space where the key point is located, which is a natural number; Loc represents the position of the key point in the image, expressed as (x,y), x is a natural number not greater than the number of pixels of the image width, and y is a natural number not greater than the number of pixels of the image length; Hes represents the Hessian matrix corresponding to a point Loc in the scale space σ of the image calculated using the SURF algorithm.

[0012] Preferably, in step S4, the image recognition module calculates His p and HIS J Similarity DH j The method is: ; In the formula, His j [k], His p [k] represents the pixel ratio corresponding to the gray value k; and His j [k], His p The mean of [k].

[0013] Preferably, in step S6, the method for identifying the comparison material whose characteristic matching value TZ is greater than the set threshold value ΔTZ as a candidate target is: S601, extracting feature information FEAT of the industrial camera image; S602, sequentially taking out the image feature feat of each comparison material whose DH is greater than the set threshold △DH, and calculating the matching number r between feat and the feature points in FEAT by using the SURF feature point matching principle; S603, among the materials for comparison whose DH is greater than the set threshold △DH, select the material for comparison with the largest number of feature point matches r to solve the feature matching value TZ. If the TZ is greater than the set threshold △TZ, it is the target comparison material: ; In the formula, r all It is an integer, indicating the total number of feature points of the compared material.

[0014] The beneficial effects of the automatic acceptance device and method for e-commerce materials of electric power enterprises based on image and data fusion provided by the present invention are as follows: This application solves the problem that the existing power companies' material acceptance and warehousing process relies heavily on manual verification and classification, which is time-consuming, labor-intensive and has poor accuracy, resulting in high acceptance costs. It has the characteristics of automatic acceptance through image recognition and classification of unqualified materials by type, which greatly reduces the degree of manual intervention and thus improves efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] Embodiment 1: like Figure 1 As shown, an automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion includes a data acquisition module, a weighing and measuring module, a logistics tracking module, a classification acceptance module and a server; The data acquisition module includes an industrial camera and a card reader. The industrial camera is used to collect multi-angle images of materials, and the card reader is used to collect logistics information of the logistics tracking module. The weighing measurement module includes a high-precision weighing sensor, which is used to collect the weight data of the materials in the warehouse; The logistics tracking module includes an RFID electronic tag set on the outer packaging of the material and a positioning module. The positioning module is used to collect the location information of the material and the corresponding timestamp to form a "location-time" electronic tag and cooperate with the card reader of the data acquisition module through the RFID electronic tag; The classification and acceptance module includes an image recognition module and an acceptance module. The image recognition module executes an image recognition algorithm to identify the broad categories of materials. The acceptance module subdivides and accepts the materials based on the results of broad category identification. The data acquisition module, weighing measurement module, logistics tracking module and classification acceptance module are all connected to the server through the communication module.

[0017] Preferably, the communication module includes an LTE communication module, a wifi module and a Bluetooth transceiver module.

[0018] Preferably, an image database, a comprehensive acquisition database and a filing database are provided in the server. The server stores image data collected by industrial cameras, the comprehensive acquisition database stores location data, time data and weighing data of materials, and the filing database is communicated with the logistics reporting system for receiving initial data of material reporting.

[0019] Preferably, the logistics reporting system is used to fill in the initial data of the transported materials before the materials are transported. The initial data includes multi-angle images of the uploaded materials, cargo weight information, and logistics distribution shipping location and delivery time.

[0020] Preferably, the image recognition algorithm is used to calculate the weighted histogram similarity DH of the collected cargo images to be identified, and match them with the images in the filing database, so as to identify the major categories of materials.

[0021] Embodiment 2: like Figure 2 As shown, a method for an automatic acceptance device for e-commerce materials of an electric power enterprise based on image and data fusion includes the following steps: S1, using an industrial camera to capture multi-angle images of materials, by moving the industrial camera to multiple shooting positions or using multiple industrial cameras set at multiple positions to perform multi-angle shooting; S2, record shooting position and histogram information His p , is the azimuth, is the pitch angle; the colleague who took the photo reads the "position-time" data of the RFID electronic tag with a card reader, and the weighing sensor collects the weight information of the current material, stores the above-mentioned multiple data in the form of a string, and records the data for each material in turn; S3, read the initial data submitted in the logistics reporting system and set the image comparison priority in the filing database: The priority is the place of shipment and weight. The data in the record database with the same place of shipment as the current place of shipment of the material to be identified is selected as the data for priority comparison and forms a data set Q. Then, the weight data in the data set Q is subtracted from the weighing data of the current material to be identified, and the multiple comparison materials in the data set Q are sorted in ascending order of the difference value. S3, the image recognition module determines the shooting direction , extract the histogram information HIS of the image features of M adjacent preset positions of the reported comparison materials in the data set Q according to the sorting order J ; S4, image recognition module calculates His p and HIS J Similarity DH j, and further solve the weighted histogram similarity DH of the current material to be identified: ; In the formula, j represents the number of the current position, S represents the area budget, represents the total geometric area of ​​the image in M ​​directions, represents the total geometric area of ​​the image after the current j-th orientation is replaced by orientation V; S5, executing steps S3 to S4 in order for the comparison materials in the data set Q, and calculating the DH values ​​of all the comparison materials; S6, comparing the calculated DH with the set threshold △DH, performing feature matching on the compared materials whose DH is greater than the set threshold △DH, and eliminating the remaining compared materials; For the comparison materials whose feature matching value TZ is greater than the set threshold △TZ, they are identified as candidate targets, and the comparison materials with the largest feature matching value TZ among multiple candidate targets are selected as target comparison materials; TZ is the ratio of feature matching data to the total number of features, TZ satisfies 0≤TZ≤1, and △TZ satisfies 0≤△TZ≤1; S7, comparing the weight information, place of shipment and delivery time reported by the comparison material selected in step S6 with the currently measured material, if all three items are the same, matching the comparison material with the current material to be identified, marking it as "accepted" and entering the warehouse according to the reported information; if the weight information is different, it is determined to be "goods quantity defect" and classified; if the delivery time is different, it is determined to be "material overdue" and classified; S8, in step S6, if there is no material to be compared whose DH is greater than the set threshold △DH, the material to be identified is classified as "abnormal" and manual comparison is performed after all materials are inspected and accepted.

[0022] Preferably, during the data processing, the material information and the acceptance results are encrypted by encryption technology.

[0023] Preferably, in step S3, the method for acquiring the multi-angle images of the materials in the filing database is: S301, from N preset positions Take images of delivered materials i And the corresponding histogram information His i , represented by His i [V0,V1,......,V255],His i is a one-dimensional array with 256 elements, V i Represents the ratio of pixels with gray value i in the image to the total pixels in the image; i is the number indicating the orientation. For preset position The azimuth of For preset position The pitch angle; S302, extracting image T i The feature information FEAT is expressed as FEAT{kp1,kp2,......,kpn}, where kp represents a key point in the image feature information, expressed as kp{σ,Loc,Hes}, σ represents the scale of the scale space where the key point is located, which is a natural number; Loc represents the position of the key point in the image, expressed as (x,y), x is a natural number not greater than the number of pixels of the image width, and y is a natural number not greater than the number of pixels of the image length; Hes represents the Hessian matrix corresponding to a point Loc in the scale space σ of the image calculated using the SURF algorithm.

[0024] Preferably, in step S4, the image recognition module calculates His p and HIS J Similarity DH j The method is: ; In the formula, His j [k], His p [k] represents the pixel ratio corresponding to the gray value k; and His j [k], His p The mean of [k].

[0025] Preferably, in step S6, the method for identifying the comparison material whose characteristic matching value TZ is greater than the set threshold value ΔTZ as a candidate target is: S601, extracting feature information FEAT of the industrial camera image; S602, sequentially taking out the image feature feat of each comparison material whose DH is greater than the set threshold △DH, and calculating the matching number r between feat and the feature points in FEAT by using the SURF feature point matching principle; S603, among the materials for comparison whose DH is greater than the set threshold △DH, select the material for comparison with the largest number of feature point matches r to solve the feature matching value TZ. If the TZ is greater than the set threshold △TZ, it is the target comparison material: ; In the formula, r all It is an integer, indicating the total number of feature points of the compared material.

Claims

1. An automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion, characterized by: Including data acquisition module, weighing measurement module, logistics tracking module, classification acceptance module and server; The data acquisition module includes an industrial camera and a card reader. The industrial camera is used to collect multi-angle images of materials, and the card reader is used to collect logistics information of the logistics tracking module. The weighing measurement module includes a high-precision weighing sensor, which is used to collect the weight data of the materials in the warehouse; The logistics tracking module includes an RFID electronic tag set on the outer packaging of the material and a positioning module. The positioning module is used to collect the location information of the material and the corresponding timestamp to form a "location-time" electronic tag and cooperate with the card reader of the data acquisition module through the RFID electronic tag; The classification and acceptance module includes an image recognition module and an acceptance module. The image recognition module executes an image recognition algorithm to identify the broad categories of materials. The acceptance module subdivides and accepts the materials based on the results of broad category identification. The data acquisition module, weighing measurement module, logistics tracking module and classification acceptance module are all connected to the server through the communication module.

2. According to claim 1, the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion is characterized by: The communication module includes an LTE communication module, a wifi module and a Bluetooth transceiver module.

3. According to claim 1, the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion is characterized by: The server is equipped with an image database, a comprehensive collection database and a filing database. The server stores image data collected by industrial cameras. The comprehensive collection database stores the location data, time data and weighing data of materials. The filing database is connected to the logistics reporting system for receiving the initial data of material reporting.

4. According to claim 3, the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion is characterized by: The logistics reporting system is used to fill in the initial data of the transported materials before transportation. The initial data includes multi-angle images of the uploaded materials, cargo weight information, and logistics distribution shipping location and delivery time.

5. According to claim 1, the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion is characterized by: The image recognition algorithm is used to calculate the weighted histogram similarity DH of the collected cargo images to be identified, and match them with the images in the filing database to identify the major categories of materials.

6. A method for automatic acceptance of e-commerce materials for electric power enterprises based on image and data fusion according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1, using an industrial camera to capture multi-angle images of materials, by moving the industrial camera to multiple shooting positions or using multiple industrial cameras set at multiple positions to perform multi-angle shooting; S2, record shooting position and histogram information His p , is the azimuth, is the pitch angle; the colleague who is shooting reads the "position-time" data of the RFID electronic tag with a card reader, and the weighing sensor collects the weight information of the current material, stores the above-mentioned multiple data in the form of a string, and records the data for each material in turn; S3, read the initial data submitted in the logistics reporting system and set the image comparison priority in the filing database: The priority is the place of shipment and weight. The data in the record database with the same place of shipment as the current place of shipment of the material to be identified is selected as the data for priority comparison and forms a data set Q. Then, the weight data in the data set Q is subtracted from the weighing data of the current material to be identified, and the multiple comparison materials in the data set Q are sorted in ascending order of the difference value. S3, the image recognition module determines the shooting direction , extract the histogram information HIS of the image features of M adjacent preset positions of the reported comparison materials in the data set Q according to the sorting order J ; S4, image recognition module calculates His p and HIS J Similarity DH j , and further solve the weighted histogram similarity DH of the current material to be identified: ； In the formula, j represents the number of the current position, S represents the area budget, represents the total geometric area of ​​the image in M ​​directions, represents the total geometric area of ​​the image after the current j-th orientation is replaced by orientation V; S5, executing steps S3 to S4 in order for the comparison materials in the data set Q, and calculating the DH values ​​of all the comparison materials; S6, comparing the calculated DH with the set threshold △DH, performing feature matching on the compared materials whose DH is greater than the set threshold △DH, and eliminating the remaining compared materials; For the comparison materials whose feature matching value TZ is greater than the set threshold △TZ, they are identified as candidate targets, and the comparison materials with the largest feature matching value TZ among multiple candidate targets are selected as target comparison materials; TZ is the ratio of feature matching data to the total number of features, TZ satisfies 0≤TZ≤1, and △TZ satisfies 0≤△TZ≤1; S7, comparing the weight information, place of shipment and delivery time reported by the comparison material selected in step S6 with the currently measured material, if all three items are the same, matching the comparison material with the current material to be identified, marking it as "accepted" and entering the warehouse according to the reported information; if the weight information is different, it is determined to be "goods quantity defect" and classified; if the delivery time is different, it is determined to be "material overdue" and classified; S8, in step S6, if there is no material to be compared whose DH is greater than the set threshold △DH, the material to be identified is classified as "abnormal" and manual comparison is performed after all materials are inspected and accepted.

7. The method of the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion according to claim 6 is characterized in that: During the data processing process, material information and acceptance results are encrypted using encryption technology.

8. The method of the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion according to claim 6 is characterized in that: In step S3, the method for acquiring the multi-angle images of the materials in the filing database is: S301, from N preset positions Take images of delivered materials i And the corresponding histogram information His i , represented by His i [V0,V1,......,V255],His i is a one-dimensional array with 256 elements, V i Represents the ratio of pixels with gray value i in the image to the total pixels in the image; i is the number indicating the orientation. For preset position The azimuth of For preset position The pitch angle; S302, extracting image T i The feature information FEAT is expressed as FEAT{kp1,kp2,......,kpn}, where kp represents a key point in the image feature information, expressed as kp{σ,Loc,Hes}, σ represents the scale of the scale space where the key point is located, which is a natural number; Loc represents the position of the key point in the image, expressed as (x,y), x is a natural number not greater than the number of pixels of the image width, and y is a natural number not greater than the number of pixels of the image length; Hes represents the Hessian matrix corresponding to a point Loc in the scale space σ of the image calculated using the SURF algorithm.

9. The method of the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion according to claim 8 is characterized in that: In step S4, the image recognition module calculates His p and HIS J Similarity DH j The method is: ； In the formula, His j [k], His p [k] represents the pixel ratio corresponding to the gray value k; and His j [k], His p The mean of [k].

10. The method of the automatic acceptance device for e-commerce materials of electric power enterprises based on image and data fusion according to claim 9 is characterized in that: In step S6, the method of identifying the comparison materials whose feature matching value TZ is greater than the set threshold ΔTZ as candidate targets is as follows: S601, extracting feature information FEAT of the industrial camera image; S602, sequentially taking out the image feature feat of each comparison material whose DH is greater than the set threshold △DH, and calculating the matching number r between feat and the feature points in FEAT by using the SURF feature point matching principle; S603, among the materials for comparison whose DH is greater than the set threshold △DH, select the material for comparison with the largest number of feature point matches r to solve the feature matching value TZ. If the TZ is greater than the set threshold △TZ, it is the target comparison material: ; In the formula, r all It is an integer, indicating the total number of feature points of the compared material.

Images