Material checking method and device, electronic equipment and storage medium

Abstract

The invention discloses a material checking method and device, electronic equipment and a storage medium. Comprising the steps that a to-be-processed image is acquired, and the to-be-processed image comprises at least one to-be-checked material; performing image recognition on the to-be-processed image based on a pre-trained image processing model, and determining an image recognition result of at least one to-be-checked material in the to-be-processed image, the image recognition result including material category information and target contour information; matching in a pre-constructed standard model library based on the material category information and / or the target contour information of each to-be-checked material, and determining standard contour information of each to-be-checked material; and based on the standard contour information of each to-be-checked material and the corresponding target contour information, determining checking data of each to-be-checked material. According to the method and the device, the images of the to-be-checked materials can be identified and processed, the materials can be checked automatically, the material checking result is determined, and the efficiency and the accuracy of material checking are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a material inventory method, device, electronic equipment and storage medium. Background Art

[0002] With the improvement of the standardization requirements of material management, the requirements for material inventory are getting higher and higher. Among them, the quantity and specifications of materials are large. For example, materials can be cables of different specifications. It is slightly easier to count unused materials, but it is difficult to estimate the size data of used materials. A lot of human resources and a lot of time are wasted in the inventory process. Due to the large number of types and quantities, it will also lead to large inventory errors, making the material inventory results inaccurate. Summary of the invention

[0003] The present invention provides a material inventory method, device, electronic equipment and storage medium to solve the problem that materials cannot be automatically counted, and the problem that material inventory is inaccurate and inefficient.

[0004] According to one aspect of the present invention, there is provided a material inventory method, comprising:

[0005] Acquire an image to be processed, where the image to be processed includes at least one material to be counted;

[0006] Performing image recognition on the image to be processed based on the pre-trained image processing model, and determining an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information;

[0007] Based on the material category information and / or target profile information of each material to be counted, matching is performed in a pre-built standard model library to determine the standard profile information of each material to be counted;

[0008] Determine the inventory data of each material to be counted based on the standard profile information of each material to be counted and the corresponding target profile information

[0009] Optionally, the pre-trained image processing model includes a feature map extraction sub-model, a classification sub-model and a boundary regression sub-model; image recognition is performed on the image to be processed based on the pre-trained image processing model to determine the image recognition result of at least one material to be counted in the image to be processed, including: extracting feature maps of the image to be processed based on the feature map extraction sub-model to obtain feature map data of at least one material to be counted in the image to be processed; classifying each feature map data based on the classification sub-model to obtain material category information of each feature map; and performing boundary adjustment processing on each feature map data based on the boundary regression sub-model to obtain target contour information of each feature map data.

[0010] Optionally, feature map extraction is performed on the image to be processed based on the feature map extraction submodel to obtain feature map data of at least one material to be counted in the image to be processed, including: processing the image to be processed based on the region extraction network to obtain candidate regions of interest for each material to be counted; and cropping the candidate regions of interest for each material to be counted to obtain feature map data of a preset size corresponding to each material to be counted.

[0011] Optionally, standard profile information of each material to be counted is determined based on matching the material category information and / or target profile information of each material to be counted in a pre-constructed standard model library, including: traversing the pre-constructed standard model library to obtain at least one material category information and standard profile information in the standard model library; determining standard profile information of each material to be counted based on matching the material category information of each material to be counted with the material category information in the standard model library; and / or determining standard profile information of each material to be counted based on matching the target profile information of each material to be counted with the standard profile information in the standard model library.

[0012] Optionally, determining the inventory data of each material to be counted based on the standard profile information of each material to be counted and the corresponding target profile information includes:

[0013] Based on the target profile information of each material to be counted, the estimated size data of each material to be counted is determined, and based on the standard profile information of each material to be counted, the standard size data of each material to be counted and the standard data of the items to be counted are determined; the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data are calculated, and the inventory data of the items to be counted of each material to be counted is determined based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted.

[0014] Optionally, after determining the image recognition result of at least one material to be counted in the image to be processed, the method further includes: performing occluded area detection processing on the image to be processed, and when it is detected that there is an occluded area in the image to be processed, performing overlapping analysis based on contour information of unoccluded areas in the image to be processed through a pre-trained image processing model to determine the image recognition result of the material to be counted in the occluded area in the image to be processed, and adding the image recognition result of the material to be counted in the occluded area in the image to be processed to the image recognition result of the image to be processed.

[0015] Optionally, after determining the image recognition result of at least one material to be counted in the image to be processed, the method further includes: performing feature map conversion processing on the target contour information in the image recognition results of each material to be counted based on a preset full convolutional network to obtain a mask result with the same size as the original image corresponding to each material to be counted; matching the material category information and / or the mask result of each material to be counted in a pre-built standard model library to determine the standard contour information of each material to be counted.

[0016] According to another aspect of the present invention, there is provided a material inventory device, comprising:

[0017] A module for acquiring images to be processed, used for acquiring images to be processed, wherein the images to be processed include at least one material to be counted;

[0018] An image recognition result determination module is used to perform image recognition on the image to be processed based on a pre-trained image processing model, and determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information;

[0019] A standard profile information determination module is used to determine the standard profile information of each material to be counted by matching the material category information and / or target profile information of each material to be counted in a pre-built standard model library;

[0020] The inventory data determination module is used to determine the inventory data of the materials to be counted based on the standard profile information of each material to be counted and the corresponding target profile information.

[0021] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the material inventory counting method of any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the material inventory method of any embodiment of the present invention when executed.

[0026] The technical solution of the embodiment of the present invention is to obtain an image to be processed, wherein the image to be processed includes at least one material to be counted; perform image recognition on the image to be processed based on a pre-trained image processing model to determine the image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; perform matching in a pre-built standard model library based on the material category information and / or target contour information of each material to be counted to determine the standard contour information of each material to be counted; and determine the inventory data of each material to be counted based on the standard contour information of each material to be counted and the corresponding target contour information. The invention realizes that the material inventory can be automatically performed and the material inventory result can be determined by performing image recognition and processing on the image of the material to be counted, so as to solve the problem that the material cannot be automatically counted, and the problem that the material inventory is inaccurate and inefficient, and improve the efficiency and accuracy of the material inventory.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a flow chart of a material inventory method provided in Embodiment 1 of the present invention;

[0030] Figure 2 is a flow chart of a material inventory method provided in Embodiment 2 of the present invention;

[0031] Figure 3 It is a structural schematic diagram of a material inventory device provided in Embodiment 3 of the present invention;

[0032] Figure 4 It is a structural schematic diagram of an electronic device for implementing the material inventory counting method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar materials, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 This is a flow chart of a material inventory method provided in the first embodiment of the present invention. This embodiment is applicable to the case of taking inventory of materials. The method can be executed by a material inventory device. The material inventory device can be implemented in the form of hardware and / or software. The material inventory device can be configured in electronic devices such as computers and terminal devices. Figure 1 As shown, the method includes:

[0037] S110: Acquire an image to be processed, where the image to be processed includes at least one material to be counted.

[0038] The image to be processed can be specifically understood as an image to be processed for image recognition processing, and the image is captured by an image capture device, and the captured image to be processed includes at least one material to be counted. The material to be counted can be specifically understood as an object to be counted, which can be an object with a certain regular shape, or an object formed by winding a linear object in a standardized manner, including but not limited to cables of various specifications.

[0039] Specifically, image acquisition equipment can be installed in advance in an environment where there is a need to count materials. The image data to be processed can be collected at a fixed time by a timing device, or a handheld camera can be used to shoot the image data to be processed when material inventory is required. It should be noted that the image to be processed needs to include at least one material to be counted. If the material to be counted is not identified in the collected image data to be processed, the image can be deleted to improve the data quality of the image to be processed.

[0040] In this embodiment, by acquiring an image to be processed containing at least one physical object to be counted, an automated material inventory should be subsequently performed, providing a data basis for the automated material inventory, thereby realizing automated processing of the material inventory.

[0041] S120. Perform image recognition on the image to be processed based on the pre-trained image processing model to determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information.

[0042] Among them, the pre-trained image processing model can be specifically understood as a model for image recognition, which can be trained using a deep learning model, including but not limited to Mask R-CNN (Mask Region-based Convolutional Neural Network) and FAST R-CNN (FAST Region-based Convolutional Neural Network). In this embodiment, Mask R-CNN is used to identify the target of the image to be processed, wherein the target is the material to be counted in the image to be processed. The material category information represents the material type to which the material to be counted belongs, and the category information of different materials is different. For example, for cables, the material category information can be the specification information of the cable. The target contour information represents the contour information segmented from the image corresponding to the material to be counted.

[0043] Specifically, the image to be processed is taken as input data and input into a pre-trained image processing model. The image processing model performs target detection and instance segmentation on the image to be processed, and identifies information of each target in the image to be processed, that is, identifies an image recognition result of at least one material to be counted in the image to be processed. The output image recognition result includes but is not limited to material category information and target contour information of the target.

[0044] Optionally, the pre-trained image processing model includes a feature map extraction sub-model, a classification sub-model and a boundary regression sub-model; image recognition is performed on the image to be processed based on the pre-trained image processing model to determine the image recognition result of at least one material to be counted in the image to be processed, including: extracting feature maps of the image to be processed based on the feature map extraction sub-model to obtain feature map data of at least one material to be counted in the image to be processed; classifying each feature map data based on the classification sub-model to obtain material category information of each feature map; and performing boundary adjustment processing on each feature map data based on the boundary regression sub-model to obtain target contour information of each feature map data.

[0045] Among them, the feature extraction sub-model can be specifically understood as a model for image feature extraction, which can be a region extraction network to extract image features, used for target detection and segmentation tasks, and determine the area in the image to be processed that may contain the materials to be counted. The region extraction network is a fully convolutional network. The classification sub-model can be specifically understood as a model for classifying targets in image feature data, and determining the material category to which each target belongs. The classification sub-model is also a fully convolutional network, in which the material type information represents the category information to which the materials to be counted belong. The material category corresponding to each material can be set according to actual needs. For example, if the cable is identified in the image to be processed, the material category information output by the classification sub-model is the specification information of the cable. The boundary regression sub-model can be specifically understood as a model for correcting the boundary of the feature map, so that the boundary box of the feature map fits the actual material to be counted more closely.

[0046] Specifically, the pre-trained image processing model includes a feature map extraction sub-model, a classification sub-model and a boundary regression sub-model, and the above sub-models are used together to complete the image processing operation on the image to be processed. The feature map extraction sub-model is used to extract the feature map of the image to be processed, and the area in the image to be processed that may contain the material to be counted is identified, and the feature map data of at least one material to be counted in the image to be processed is obtained; the feature map data are respectively input into the classification sub-model and the boundary regression sub-model for processing, and the material category information corresponding to each feature map data and the target contour information of each feature map data are output.

[0047] In this embodiment, image recognition is performed on the image to be processed through a pre-trained image processing model to identify the material category information and target contour information of each material to be counted in the image to be processed, which is used for subsequent determination of the inventory data of each material to be counted, thereby helping to realize the automated processing of material inventory, improve the efficiency and accuracy of material inventory, and reduce the waste of human resources and time.

[0048] For the training of the image processing model, the feature map extraction sub-model, classification sub-model and boundary regression sub-model are jointly trained by obtaining training sample data and a pre-built standard model library. Corresponding loss functions are set for the feature map extraction sub-model, classification sub-model and boundary regression sub-model respectively. The FCN algorithm is used to predict the feature map extraction sub-model, and then the prediction error loss function is determined. The multi-category cross entropy loss function is used for the classification sub-model. The boundary regressor is used to calculate the loss value of the bounding box for the boundary regression sub-model. Then, the sum of the loss values ​​of multiple sub-models is calculated to obtain the overall loss value of the image processing model, which is used to train the evaluation model. When the training results meet the preset training end conditions, the trained image processing model is obtained.

[0049] Optionally, feature map extraction is performed on the image to be processed based on the feature map extraction submodel to obtain feature map data of at least one material to be counted in the image to be processed, including: processing the image to be processed based on the region extraction network to obtain candidate regions of interest for each material to be counted; and cropping the candidate regions of interest for each material to be counted to obtain feature map data of a preset size corresponding to each material to be counted.

[0050] The image to be processed is processed by the regional feature extraction network. According to the feature map in the image to be processed, a series of anchor boxes are generated according to a certain rule. Assuming that the size of the feature map is W×H, W×H×k anchor boxes are generated, that is, k anchor boxes are generated for each point on the feature map. Specifically, a small sliding window filter (usually 3×3) is slid on the feature map, and the foreground / background scores and bounding box regression parameters of multiple anchors (Anchor) at each position are predicted. Anchors are rectangular boxes of fixed size and proportion, and they exist at every position on the feature map. The regional feature extraction network selects the region most likely to contain the target object from these anchors. The regional feature extraction network outputs a set of bounding boxes as candidate regions, extracts features from each candidate region, and crops the candidate regions of interest of each material to be counted by the region of interest pooling method or the region of interest alignment method to obtain the feature map data of the preset size corresponding to each material to be counted, so as to ensure that the feature map corresponding to each region of interest meets the preset size, so as to facilitate further classification and regression processing.

[0051] On the basis of the above embodiment, after determining the image recognition result of at least one material to be counted in the image to be processed, the method also includes: performing occluded area detection processing on the image to be processed, and when it is detected that there is an occluded area in the image to be processed, performing overlapping analysis based on the contour information of the unoccluded area in the image to be processed through a pre-trained image processing model to determine the image recognition result of the material to be counted in the occluded area in the image to be processed, and adding the image recognition result of the material to be counted in the occluded area in the image to be processed to the image recognition result of the image to be processed.

[0052] It should be noted that when the materials to be counted are stored, there may be a situation where the materials to be counted are blocked. Therefore, when the materials to be counted in the image to be processed are determined by performing image recognition on the image to be processed, it is necessary to consider identifying the materials to be counted in the blocked area to ensure that the materials to be counted in the image to be processed can be completely identified. The pre-trained image processing model can also specifically detect the blocked area through the position information of the identified materials to be counted. When the blocked area is detected in the image to be processed, the image information estimation sub-model in the pre-trained image processing model performs overlapping analysis based on the contour information of the unblocked area in the image to be processed to estimate the image recognition result of the materials to be counted in the blocked area in the image to be processed. Then, the image recognition result of the materials to be counted in the blocked area in the image to be processed is added to the image recognition result of the image to be processed, thereby obtaining the complete recognition result of the materials to be counted in the image to be processed. The region extraction network can generate multiple overlapping candidate regions to capture different parts of the occluded object. When dealing with occlusion, an additional occlusion branch is introduced, which is responsible for detecting and identifying occluded areas in the image, thereby helping the network to better locate and segment occluded targets. For each region of interest, the pre-trained image processing model predicts its category and bounding box regression parameters. These parameters are used to adjust the position and size of the region of interest to more accurately frame the target object. Even if part of the target is occluded, the pre-trained image processing model can still use the unoccluded part to generate a segmentation mask. Even if part of the target is occluded, the bounding box regression can still try to correct the position and size of the bounding box to make it as consistent as possible with the actual target position. For example, even if part of an object is occluded, the model can use the features of the unoccluded part to infer the existence of the entire object and try to segment out the occluded part.

[0053] S130: Matching is performed in a pre-built standard model library based on the material category information and / or target profile information of each material to be counted, so as to determine the standard profile information of each material to be counted.

[0054] Among them, the pre-constructed standard model library is constructed in advance by collecting and identifying images of various types of materials to be counted. The images of the materials to be counted can be calibrated to obtain characteristic information of various material types, including but not limited to specification information, height, length, width and standard contour information, that is, the information stored in the pre-constructed standard model library includes but is not limited to material category information, height, length, width, standard contour information and the mask results corresponding to the standard contour information.

[0055] Specifically, the material category information of each material to be counted and one or an object in the target profile information can be matched in a pre-built standard model library respectively, and the matched standard profile information is set as the standard profile information of the corresponding material to be counted, thereby obtaining the standard profile information of each material to be counted.

[0056] Optionally, standard profile information of each material to be counted is determined based on matching the material category information and / or target profile information of each material to be counted in a pre-constructed standard model library, including: traversing the pre-constructed standard model library to obtain at least one material category information and standard profile information in the standard model library; determining standard profile information of each material to be counted based on matching the material category information of each material to be counted with the material category information in the standard model library; and / or determining standard profile information of each material to be counted based on matching the target profile information of each material to be counted with the standard profile information in the standard model library.

[0057] Specifically, the pre-constructed standard model library is traversed to obtain at least one material category information and standard profile information in the standard model library. The material category information of each material to be counted can be matched in the pre-constructed standard model library, and the standard profile information corresponding to the successfully matched material category information is set as the standard profile information of the corresponding material to be counted. Alternatively, the target profile information of each material to be counted can be matched in the pre-constructed standard model library, and the standard profile information corresponding to the successfully matched target profile information is set as the standard profile information of the corresponding material to be counted. Alternatively, by matching the material category information and the target profile information of each material to be counted in the pre-constructed standard model library respectively, two standard profile information of each material to be counted can be obtained, and the similarity of the two standard profile information can be calculated. When the similarity of the two corresponding standard profile information of each material to be counted meets the similarity threshold, the two standard profile information are merged to obtain the standard profile information of each material to be counted. When the similarity of the two corresponding standard profile information of each material to be counted does not meet the similarity threshold, the standard profile information corresponding to the successfully matched target profile information can be set as the standard profile information of the corresponding material to be counted.

[0058] S140: Determine the inventory data of each material to be counted based on the standard profile information of each material to be counted and the corresponding target profile information.

[0059] The inventory data may be specifically understood as data of items to be counted of materials to be counted, and the items to be counted include but are not limited to information on the length, quantity and weight of the materials to be counted, and may be specifically set according to the inventory requirements of the materials to be counted.

[0060] Specifically, the ratio between the standard contour information and the target contour information can be determined by comparing the standard contour information of each material to be counted with the corresponding target contour information. For example, the ratio of the contour areas corresponding to the standard contour information and the target contour information can be calculated, or the ratio of the heights corresponding to the standard contour information and the target contour information can be calculated, or the ratio of the widths corresponding to the standard contour information and the target contour information can be calculated, or the ratio of the lengths corresponding to the standard contour information and the target contour information can be calculated, and then the corresponding inventory data can be determined based on the ratio and the standard numerical value of the item to be counted corresponding to the standard contour information.

[0061] On the basis of the above embodiment, after determining the image recognition result of at least one material to be counted in the image to be processed, the method further includes: performing feature map conversion processing on the target contour information in the image recognition result of each material to be counted based on a preset full convolutional network to obtain a mask result with the same size as the original image corresponding to each material to be counted; matching the material category information and / or the mask result of each material to be counted in a pre-built standard model library to determine the standard contour information of each material to be counted.

[0062] The feature map conversion process specifically refers to converting the target contour into a pixel-level mask of the same size as the original image of the corresponding material to be counted, and the target contour information can be converted through a full convolutional network. The size of the preset full convolutional network is set based on the corresponding original image size.

[0063] Specifically, the target contour information in the image recognition results of each material to be counted is subjected to feature map conversion processing by using a preset full convolutional network, and the target contour of each material to be counted is converted into a mask result with the same size as the corresponding original image, so as to obtain a pixel-level mask corresponding to each material to be counted, so as to more accurately segment the boundaries of the material to be counted. The material category information of each material to be counted can be matched in a pre-constructed standard model library, and the standard contour information corresponding to the successfully matched material category information can be set as the standard contour information of the corresponding material to be counted. Alternatively, the mask result of each material to be counted can be matched in a pre-constructed standard model library, and the standard contour information corresponding to the successfully matched mask result can be set as the standard contour information of the corresponding material to be counted. Alternatively, by matching the material category information and mask results of each material to be counted in a pre-built standard model library, two standard profile information of each material to be counted can be obtained, and the similarity of the two standard profile information can be calculated. When the similarity of the two corresponding standard profile information of each material to be counted meets the similarity threshold, the two standard profile information are fused to obtain the standard profile information of each material to be counted. When the similarity of the two corresponding standard profile information of each material to be counted does not meet the similarity threshold, the standard profile information corresponding to the successfully matched target profile information can be set as the standard profile information of the corresponding material to be counted.

[0064] Based on the above embodiment, the method further includes: performing statistics on each material to be counted based on the standard profile data of each material to be counted, determining the quantity of the material to be counted corresponding to the same standard profile data, and obtaining the quantity of the material to be counted.

[0065] The technical solution of this embodiment is to obtain an image to be processed, wherein the image to be processed includes at least one material to be counted; perform image recognition on the image to be processed based on a pre-trained image processing model to determine the image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; perform matching in a pre-built standard model library based on the material category information and / or target contour information of each material to be counted to determine the standard contour information of each material to be counted; and determine the inventory data of each material to be counted based on the standard contour information of each material to be counted and the corresponding target contour information. The invention realizes that the material inventory can be automatically counted and the material inventory result can be determined by performing image recognition and processing on the image of the material to be counted, so as to solve the problem that the material cannot be automatically counted, and the problem that the material inventory is inaccurate and inefficient, and improve the efficiency and accuracy of the material inventory.

[0066] Embodiment 2

[0067] Figure 2It is a flow chart of a material inventory method provided by the second embodiment of the present invention. This embodiment is a further optimization of the method of the above embodiment. Optionally, the estimated size data of each material to be counted is determined based on the target profile information of each material to be counted, and the standard size data of each material to be counted and the standard data of the items to be counted are determined based on the standard profile information of each material to be counted; the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data are calculated, and the inventory data of the items to be counted of each material to be counted is determined based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted. Figure 2 As shown, the method includes:

[0068] S210: Acquire an image to be processed, where the image to be processed includes at least one material to be counted.

[0069] S220. Perform image recognition on the image to be processed based on the pre-trained image processing model to determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information.

[0070] S230: Matching is performed in a pre-built standard model library based on the material category information and / or target profile information of each material to be counted, to determine the standard profile information of each material to be counted.

[0071] S240: Determine estimated size data of each material to be counted based on target profile information of each material to be counted, and determine standard size data of each material to be counted and standard data of items to be counted based on standard profile information of each material to be counted.

[0072] Among them, the estimated size data can be specifically understood as the pixel-level size data that characterizes the target contour of the material to be counted, and the estimated size data includes but is not limited to pixel-level length data, pixel-level height data, and pixel-level area data. The standard data can be specifically understood as the standard data of a complete material to be counted, which can include but is not limited to length data and weight data. The items to be counted can be set according to the material inventory requirements, and can include but is not limited to material length and material weight.

[0073] Specifically, the pixel-level size of the material to be counted is estimated according to the target contour information of each material to be counted, which may be the length dimension, height dimension or area dimension. The mapping relationship between the standard contour information and the standard dimension data is called to determine the standard dimension data that matches the standard contour information of each material to be counted. The mapping relationship between the standard contour information and the standard data of the item to be counted is called to determine the standard data that matches the standard contour information of each material to be counted.

[0074] S250, calculating the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data, and determining the inventory data of the items to be counted of each material to be counted based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted.

[0075] Specifically, by calculating the estimated size data of each material to be counted and dividing it by the corresponding standard size data, the size ratio data of each material to be counted is obtained, and then the product of the size ratio data of each material to be counted and the standard data of the corresponding item to be counted is calculated to obtain the inventory data of the item to be counted of each material to be counted. Exemplarily, for the length data of the counted cables, the estimated height data of each bundle of cables is calculated as h, the standard size data of each bundle of cables is obtained as H, the size ratio data of a bundle of cables of this type is h / H, the item to be counted is the length data of the cable, and the standard data of the length of each bundle of cables is L, then the inventory data of the length of each bundle of cables is (h / H)*L.

[0076] The technical solution of this embodiment is as follows: by acquiring an image to be processed, the image to be processed includes at least one material to be counted; performing image recognition on the image to be processed based on a pre-trained image processing model to determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; matching the material category information and / or target contour information of each material to be counted in a pre-built standard model library to determine the standard contour information of each material to be counted; determining the estimated size data of each material to be counted based on the target contour information of each material to be counted, and determining the standard size data of each material to be counted and the standard data of the items to be counted based on the standard contour information of each material to be counted; calculating the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data, and determining the inventory data of the items to be counted of each material to be counted based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted. It is possible to automatically count materials by identifying and processing images of materials to be counted, determine the inventory data of the items to be counted of each material to be counted, and obtain the inventory results, so as to solve the problem of being unable to automatically count materials, as well as the problems of inaccurate and low efficiency in material inventory counting, and improve the efficiency and accuracy of material inventory counting.

[0077] Embodiment 3

[0078] Figure 3 Schematic diagram of the structure of a material inventory device provided by the third embodiment of the present invention. Figure 3 As shown, the device comprises:

[0079] The to-be-processed image acquisition module 310 is used to acquire an to-be-processed image, wherein the to-be-processed image includes at least one material to be counted;

[0080] An image recognition result determination module 320 is used to perform image recognition on the image to be processed based on a pre-trained image processing model, and determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information;

[0081] A standard profile information determination module 330 is used to determine the standard profile information of each material to be counted by matching the material category information and / or target profile information of each material to be counted in a pre-built standard model library;

[0082] The inventory data determination module 340 is used to determine the inventory data of the materials to be counted based on the standard profile information of each material to be counted and the corresponding target profile information.

[0083] The technical solution of this embodiment is to obtain an image to be processed through an image acquisition module to be processed, wherein the image to be processed includes at least one material to be counted; the image recognition result determination module performs image recognition on the image to be processed based on a pre-trained image processing model, and determines the image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; the standard contour information determination module matches the material category information and / or target contour information of each material to be counted in a pre-built standard model library, and determines the standard contour information of each material to be counted; the inventory data determination module determines the inventory data of the material to be counted based on the standard contour information of each material to be counted and the corresponding target contour information. It is realized that the material inventory can be automatically counted and the material inventory result can be determined by performing image recognition and processing on the image of the material to be counted, so as to solve the problem that the material cannot be automatically counted, and the problem that the material inventory is inaccurate and inefficient, and improve the efficiency and accuracy of the material inventory.

[0084] On the basis of the above embodiment, optionally, the pre-trained image processing model includes a feature map extraction sub-model, a classification sub-model and a boundary regression sub-model; the image recognition result determination module 320 includes a feature map extraction unit, a material category information determination unit and a target contour information determination unit; the feature map extraction unit is used to perform feature map extraction on the image to be processed based on the feature map extraction sub-model to obtain feature map data of at least one material to be counted in the image to be processed; the material category information determination unit is used to classify each feature map data based on the classification sub-model to obtain material category information of each feature map; the target contour information determination unit is used to perform boundary adjustment processing on each feature map data based on the boundary regression sub-model to obtain target contour information of each feature map data.

[0085] Optionally, the image recognition result determination module 320 is specifically used to process the image to be processed based on the region extraction network to obtain the candidate region of interest of each material to be counted; crop the candidate region of interest of each material to be counted to obtain feature map data of a preset size corresponding to each material to be counted.

[0086] Optionally, the standard profile information determination module 330 is specifically used to traverse a pre-built standard model library to obtain at least one material category information and standard profile information in the standard model library; based on matching the material category information of each material to be inventoried with the material category information in the standard model library, determine the standard profile information of each material to be inventoried; and / or, based on matching the target profile information of each material to be inventoried with the standard profile information in the standard model library, determine the standard profile information of each material to be inventoried.

[0087] Optionally, the inventory data determination module 340 is specifically used to determine the estimated size data of each material to be counted based on the target profile information of each material to be counted, and to determine the standard size data of each material to be counted and the standard data of the items to be counted based on the standard profile information of each material to be counted; calculate the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data, and determine the inventory data of the items to be counted of each material to be counted based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted.

[0088] Optionally, after determining the image recognition result of at least one material to be counted in the image to be processed, the device is also used to perform occluded area detection processing on the image to be processed. When an occluded area is detected in the image to be processed, an overlapping analysis is performed based on the contour information of the unoccluded area in the image to be processed through a pre-trained image processing model to determine the image recognition result of the material to be counted in the occluded area in the image to be processed, and the image recognition result of the material to be counted in the occluded area in the image to be processed is added to the image recognition result of the image to be processed.

[0089] Optionally, after determining the image recognition result of at least one material to be counted in the image to be processed, the device is also used to perform feature map conversion processing on the target contour information in the image recognition results of each material to be counted based on a preset full convolutional network, and obtain a mask result with the same size as the original image corresponding to each material to be counted; based on the material category information and / or mask result of each material to be counted, matching is performed in a pre-built standard model library to determine the standard contour information of each material to be counted.

[0090] The material inventory counting device provided in the embodiment of the present invention can execute the material inventory counting method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0091] Embodiment 4

[0092] Figure 4 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0093] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0094] At least one component in the electronic device 10 is connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0095] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a material inventory method.

[0096] In some embodiments, the material inventory method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or at least one step of the material inventory method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the material inventory method in any other appropriate manner (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or at least one computer program, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The computer program for implementing the material inventory method of the present invention can be written in any combination of one or at least one programming language. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed completely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or completely on a remote machine or server.

[0099] Embodiment 5

[0100] Embodiment 5 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a material inventory method, the method comprising:

[0101] Acquire an image to be processed, where the image to be processed includes at least one material to be counted;

[0102] Performing image recognition on the image to be processed based on the pre-trained image processing model, and determining an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information;

[0103] Based on the material category information and / or target profile information of each material to be counted, matching is performed in a pre-built standard model library to determine the standard profile information of each material to be counted;

[0104] The inventory data of each material to be counted is determined based on the standard profile information of each material to be counted and the corresponding target profile information.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or at least one line, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A material inventory method, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes at least one material to be counted; Performing image recognition on the image to be processed based on a pre-trained image processing model to determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; Based on the material category information and / or the target profile information of each material to be counted, matching is performed in a pre-built standard model library to determine the standard profile information of each material to be counted; The inventory data of each of the materials to be counted is determined based on the standard profile information of each of the materials to be counted and the corresponding target profile information.

2. The method according to claim 1, characterized in that The pre-trained image processing model includes a feature map extraction sub-model, a classification sub-model and a boundary regression sub-model; The performing image recognition on the image to be processed based on the pre-trained image processing model to determine the image recognition result of at least one material to be counted in the image to be processed includes: Performing feature map extraction on the image to be processed based on the feature map extraction sub-model to obtain feature map data of at least one material to be counted in the image to be processed; Classify the characteristic graph data based on the classification sub-model to obtain material category information of each characteristic graph; Based on the boundary regression sub-model, boundary adjustment processing is performed on each of the feature map data to obtain target contour information of each of the feature map data.

3. The method according to claim 2, characterized in that The step of extracting a feature graph from the image to be processed based on the feature graph extraction sub-model to obtain feature graph data of at least one material to be counted in the image to be processed includes: Processing the image to be processed based on a region extraction network to obtain candidate regions of interest for each of the materials to be counted; The candidate regions of interest of each of the materials to be counted are cropped to obtain feature map data of a preset size corresponding to each of the materials to be counted.

4. The method according to claim 1, characterized in that: The matching of the material category information and / or the target profile information of each material to be counted in a pre-built standard model library to determine the standard profile information of each material to be counted includes: Traversing the pre-constructed standard model library to obtain at least one material category information and standard profile information in the standard model library; determining the standard profile information of each material to be counted based on matching the material category information of each material to be counted with the material category information in the standard model library; and / or determining the standard profile information of each material to be counted based on matching the target profile information of each material to be counted with the standard profile information in the standard model library.

5. The method according to claim 1, characterized in that The determining the inventory data of each of the materials to be counted based on the standard profile information of each of the materials to be counted and the corresponding target profile information includes: Determine estimated size data of each material to be counted based on target profile information of each material to be counted, and determine standard size data of each material to be counted and standard data of items to be counted based on standard profile information of each material to be counted; Calculate the size ratio data of the estimated size data of each material to be counted and the corresponding standard size data, and determine the inventory data of the items to be counted of each material to be counted based on the size ratio data of each material to be counted and the corresponding standard data of the items to be counted.

6. The method according to claim 1, characterized in that After determining the image recognition result of at least one material to be counted in the image to be processed, the method further includes: An occluded area detection process is performed on the image to be processed. When an occluded area is detected in the image to be processed, an overlapping analysis is performed through the pre-trained image processing model based on contour information of unoccluded areas in the image to be processed, and an image recognition result of the material to be counted in the occluded area in the image to be processed is determined, and the image recognition result of the material to be counted in the occluded area in the image to be processed is added to the image recognition result of the image to be processed.

7. The method according to claim 1 or 6, characterized in that: After determining the image recognition result of at least one material to be counted in the image to be processed, the method further includes: Based on a preset full convolutional network, feature map conversion processing is performed on the target contour information in the image recognition results of each of the materials to be counted, so as to obtain a mask result with the same size as the original image corresponding to each of the materials to be counted; Based on the material category information of each of the materials to be counted and / or the mask result, matching is performed in a pre-constructed standard model library to determine the standard profile information of each of the materials to be counted.

8. A material inventory device, characterized in that: include: A module for acquiring images to be processed, used for acquiring images to be processed, wherein the images to be processed include at least one material to be counted; An image recognition result determination module is used to perform image recognition on the image to be processed based on a pre-trained image processing model, and determine an image recognition result of at least one material to be counted in the image to be processed, wherein the image recognition result includes material category information and target contour information; A standard profile information determination module, used to determine the standard profile information of each material to be counted by matching the material category information of each material to be counted and / or the target profile information in a pre-built standard model library; The inventory data determination module is used to determine the inventory data of the materials to be counted based on the standard profile information of each material to be counted and the corresponding target profile information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the material inventory counting method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the material inventory method described in any one of claims 1 to 7 when executed.

Images