Line edge warehouse material checking method and device
By extracting and fusing the edge information of the initial image and building an inventory model, the inventory problem of irregular storage of material in line edge libraries is solved, and accurate and efficient material inventory is achieved.
Patent Information
- Application Number
- CN202411931143.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-04
AI Technical Summary
The prior art is difficult to accurately count the line edge storage materials in the case of irregular storage, different specifications, messy background environment, and occlusion.
By obtaining the initial image set of line edge library materials, the edge information of each initial image is extracted, and fused with the initial image to obtain the fused image. Then, the inventory model is constructed based on the initial image collection, and the fused image is input to the inventory model to obtain the target image. Calculate the inventory results based on the material frame and confidence in the target image.
Accurate inventory of materials in line edge warehouses in case of irregular cargo storage and occlusion, improving the accuracy and efficiency of inventory.
Smart Images
Figure CN119763086A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine vision technology, and in particular to a method and device for counting materials in a lineside warehouse. Background Art
[0002] Lineside warehouses play a vital role in modern manufacturing and logistics management. By placing materials directly near the production line, they effectively improve production efficiency, reduce production interruptions caused by material shortages, and ensure the continuous operation of the production line. Inventory counting at lineside warehouses is a key link in ensuring smooth production processes and supply chain efficiency. Through regular inventory counting, companies can ensure the accuracy of inventory data, prevent material shortages, reduce excess inventory, and reduce costs. At the same time, inventory counting helps to improve the efficiency of material management, promptly detect and deal with lost, expired or damaged materials, optimize inventory levels, and support the rational formulation of production plans. The existing inventory counting methods mainly include manual inventory counting, inventory counting by using RFID tags, and inventory counting by optical image transmission systems combined with artificial intelligence algorithm decisions.
[0003] However, manual inventory is often time-consuming and error-prone due to its over-reliance on manpower, especially in large warehouses. It is easy for human factors such as fatigue or negligence to affect the accuracy of inventory data, thereby reducing production efficiency. At the same time, manual inventory is labor-intensive, data updates slowly, and it is difficult to respond quickly in emergency situations. Although RFID inventory has improved efficiency and accuracy, the RFID tags used are expensive and face challenges such as tag damage, privacy and security issues, and signal interference. In addition, although the existing inventory method that combines optical images with artificial intelligence algorithms can reduce costs and achieve a certain degree of accuracy, it is mainly aimed at the situation where goods are stacked more regularly in stereoscopic warehouses and the difference in the shape and size of goods is small. This method still cannot solve the situation where goods are stored irregularly, with different specifications, cluttered background environment, and there are occlusions. Summary of the invention
[0004] In order to solve the above-mentioned problem that the goods are stored irregularly and cannot be accurately counted in the lineside warehouse due to obstruction, the embodiment of the present application provides a lineside warehouse material inventory method and device, and its technical solution is as follows: In a first aspect, an embodiment of the present application provides a method for counting materials in a lineside warehouse, including: Acquire an initial image set of materials in the line side warehouse; wherein the initial image set includes at least two initial images; Extracting edge information of each initial image, and fusing the edge information of each initial image with the initial image to obtain a fused image; Building an inventory model based on the initial image set, and inputting the fused image into the inventory model to obtain the target image; The counting result is calculated according to the material frame in the target image and the confidence of the material frame; wherein the counting result is the number of materials included in the initial image.
[0005] In an optional solution of the first aspect, extracting edge information of each initial image includes: Calculating the gradient amplitude of each initial image in the horizontal direction by a preset operator to obtain a first gradient amplitude; Calculating the gradient amplitude of each initial image in the vertical direction by a preset operator to obtain a second gradient amplitude; The edge information of each initial image is obtained according to the first gradient amplitude and the second gradient amplitude.
[0006] In another optional solution of the first aspect, edge information of each initial image is fused with the initial image to obtain a fused image, including: Calculate the pixel variance of each initial image; When it is detected that the pixel variance is greater than a preset first threshold, weighted fusion processing is performed on the edge information of the initial image corresponding to the pixel variance and the initial image to obtain a fused image.
[0007] In yet another optional solution of the first aspect, fusing the edge information of each initial image with the initial image to obtain a fused image further includes: Calculate the contrast value of each initial image according to a preset formula; When it is detected that the contrast value is less than a preset second threshold, edge information of the initial image corresponding to the contrast value and the initial image are superimposed and fused to obtain a fused image.
[0008] In yet another optional solution of the first aspect, the inventory model includes a feature extraction network and a feature fusion network; Input the fused image into the inventory model to obtain the target image, including: The fused image is input into the inventory model, and features are extracted from the fused image through the feature extraction network and the preset attention mechanism to obtain a feature map; According to the feature map, the preset attention mechanism and the feature fusion network, the target image is obtained.
[0009] In another optional solution of the first aspect, obtaining a target image according to the feature map, a preset attention mechanism, and a feature fusion network includes: Calculate the attention weight value corresponding to the feature map according to the preset attention mechanism; The feature map is fused through the feature fusion network and the attention weight value to obtain the target image.
[0010] In yet another optional solution of the first aspect, calculating the inventory result according to the material frame in the target image and the confidence of the material frame includes: When the confidence level of the material box in the target image is higher than a preset third threshold, the material box is included in the inventory result; wherein the preset third threshold is a positive number less than 1, and the confidence level is the probability that the material box contains the material, which is calculated by the inventory model.
[0011] In a second aspect, an embodiment of the present application provides a line-side warehouse material inventory device, comprising: The first processing module is used to obtain an initial image set of materials in the line warehouse; wherein the initial image set includes at least two initial images; The second processing module is used to extract edge information of each initial image, and fuse the edge information of each initial image with the initial image to obtain a fused image; A third processing module is used to build an inventory model based on the initial image set, and input the fused image into the inventory model to obtain a target image; The fourth processing module is used to calculate the inventory result according to the material frame in the target image and the confidence of the material frame; wherein the inventory result is the number of materials included in the initial image.
[0012] In a third aspect, an embodiment of the present application further provides a line-side warehouse material inventory device, including a processor and a memory; The processor is connected to the memory; A memory for storing executable program codes; The processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the lineside warehouse material inventory method provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect.
[0013] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the lineside warehouse material inventory method provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect can be implemented.
[0014] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least: During the inventory counting process of online warehouse materials, the edge information of the initial image is extracted and fused with the initial image to obtain a fused image that highlights the contour and shape of the object. The fused image is then input into the inventory counting model to obtain the target image output by the model, which includes the material frame and the confidence of the material frame. The inventory result is calculated based on the target image, thus achieving accurate inventory counting of online warehouse materials in the case of irregular storage and occlusion of goods. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0016] Figure 1 An overall flow chart of a lineside warehouse material inventory method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a line-side warehouse material inventory device provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of another line-side warehouse material inventory device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0018] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, although the embodiment may not be clearly recorded in the following text.
[0019] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.
[0020] See also Figure 1 , Figure 1 An overall flow chart of a lineside warehouse material inventory method provided in an embodiment of the present application is shown.
[0021] like Figure 1 As shown, the lineside warehouse material inventory method may at least include the following steps: Step 101: Obtain an initial image set of materials in the line side warehouse.
[0022] In the embodiment of the present application, the lineside warehouse material inventory method can be applied to, but is not limited to, intelligent storage devices, handheld inventory terminals, automated warehouse systems, and lineside warehouse management modules in MES systems. In the process of counting lineside warehouse materials, the edge information of the initial image is extracted, and the edge information is fused with the initial image to obtain a fused image that highlights the outline and shape of the object, and then the fused image is input into the inventory model to obtain a target image output by the model that includes a material frame and the confidence of the material frame, thereby calculating the inventory result based on the target image, and realizing accurate inventory of lineside warehouse materials in the case of irregular storage of goods and occlusion.
[0023] Specifically, during the inventory of materials in the online warehouse, an initial image set of the materials in the online warehouse can be obtained first, wherein the initial image set includes at least two initial images, and the acquisition method can be but is not limited to reading the physical images of the materials in the online warehouse taken by devices such as cameras or mobile phones and stored in the management platform.
[0024] Step 102: extract edge information of each initial image, and fuse the edge information of each initial image with the initial image to obtain a fused image.
[0025] Specifically, after obtaining the initial image, the edge information of each initial image can be extracted by the Sobel operator, and then the extracted edge information can be fused with the initial image to introduce interpretable information for the subsequent inventory model, so as to help the inventory model to adaptively adjust the network's response to these features according to the different shapes and sizes of the object, so as to better capture the accurate position of the target. It can be understood that the Sobel operator is an edge detection operator widely used in digital image processing and computer vision. It is mainly used to detect edges in images by changes in image brightness. The Sobel operator is based on the first-order derivative of the image and realizes edge detection by calculating the gradient amplitude of the image in the horizontal and vertical directions.
[0026] It should be noted that the fusion method may be but is not limited to using a mask to superimpose edge information on the original image or converting the edge image into the same color space as the original image and fusion them together using a weighted sum method.
[0027] As an option of the embodiment of the present application, extracting edge information of each initial image includes: Calculating the gradient amplitude of each initial image in the horizontal direction by a preset operator to obtain a first gradient amplitude; Calculating the gradient amplitude of each initial image in the vertical direction by a preset operator to obtain a second gradient amplitude; The edge information of each initial image is obtained according to the first gradient amplitude and the second gradient amplitude.
[0028] Specifically, in the process of extracting edge information of each initial image, the gradient amplitude (ie, the first gradient amplitude) of each initial image in the horizontal direction can be first calculated by a preset operator (ie, the Sobel operator), and the steps are as follows: (1) Reading the image: Convert the initial image to a grayscale image and use the cv2.imread() function in the OpenCV library to read the grayscale image; (2) Apply the Sobel operator: Use the cv2.Sobel() function in the OpenCV library to calculate the first-order derivative of the image in the x direction (horizontal direction). This function returns an image of the same size as the input image, in which the pixel value represents the gradient of the position in the horizontal direction; (3) Calculate the gradient magnitude: Since the Sobel operator returns gradient values, these values may be positive, negative, or zero. In order to visualize the gradient, these gradient values need to be converted into unsigned 8-bit integers in the range of 0-255.
[0029] Next, the gradient amplitude (i.e., the second gradient amplitude) of each initial image in the vertical direction can be calculated by the Sobel operator, and the steps are as follows: (1) Reading the image: Convert the initial image to a grayscale image and use the cv2.imread() function in the OpenCV library to read the grayscale image; (2) Apply the Sobel operator: Use the cv2.Sobel() function in the OpenCV library to calculate the first-order derivative of the image in the y direction (vertical direction). By setting dx=0 and dy=1, you can specify the gradient in the vertical direction that needs to be calculated.
[0030] (3) Calculate the gradient magnitude: Since the Sobel operator returns gradient values, these values may be positive, negative, or zero. In order to visualize the gradient, these gradient values need to be converted into unsigned 8-bit integers in the range of 0-255.
[0031] After calculating the first gradient magnitude and the second gradient magnitude, the two gradient magnitudes may be combined to obtain edge information of the initial image, such as marking pixels with gradient magnitudes higher than a threshold as edges, and marking pixels with gradient magnitudes lower than the threshold as non-edges.
[0032] As another option of the embodiment of the present application, edge information of each initial image is fused with the initial image to obtain a fused image, including: Calculate the pixel variance of each initial image; When it is detected that the pixel variance is greater than a preset first threshold, weighted fusion processing is performed on the edge information of the initial image corresponding to the pixel variance and the initial image to obtain a fused image.
[0033] Specifically, in the process of fusing the edge information of each initial image with the initial image to obtain a fused image, the pixel variance of each initial image can be first calculated by the cv2.meanStdDev function or the np.var function. When it is detected that the pixel variance is greater than the preset first threshold, it means that the image contains more details or edge information. At this time, it is appropriate to use a weighted fusion method to fuse the edge information and the initial image, so as to obtain a fused image that can highlight the contour and shape of the object.
[0034] As another option of the embodiment of the present application, edge information of each initial image is fused with the initial image to obtain a fused image, further comprising: Calculate the contrast value of each initial image according to a preset formula; When it is detected that the contrast value is less than a preset second threshold, edge information of the initial image corresponding to the contrast value and the initial image are superimposed and fused to obtain a fused image.
[0035] Specifically, in the process of fusing the edge information of each initial image with the initial image to obtain a fused image, the contrast value of each initial image can also be calculated according to a preset formula. When the contrast value is detected to be less than the preset second threshold, it means that the image lacks significant light and dark differences, and the details and edges in the image may become blurred and difficult to distinguish. At this time, it is appropriate to fuse the edge information and the initial image by superposition fusion, so as to obtain a fused image that can highlight the contour and shape of the object.
[0036] Step 103: construct an inventory model based on the initial image set, and input the fused image into the inventory model to obtain a target image.
[0037] Specifically, after obtaining the fused image, the initial image set can be divided into a training set, a validation set, and a test set in a ratio of 8:1:1, and an inventory model can be trained based on the EA-YOLO network model.
[0038] Furthermore, after obtaining the inventory model, the fused image can be input into the inventory model to obtain the target image output by the model. It can be understood that the fused image is used as the input data of the model, and the target image is used as the output data of the model, wherein the target image includes the material box recognized by the inventory model and the corresponding confidence.
[0039] As another option of the embodiment of the present application, the inventory model includes a feature extraction network and a feature fusion network; Input the fused image into the inventory model to obtain the target image, including: The fused image is input into the inventory model, and features are extracted from the fused image through the feature extraction network and the preset attention mechanism to obtain a feature map; According to the feature map, the preset attention mechanism and the feature fusion network, the target image is obtained.
[0040] Specifically, the inventory model includes a feature extraction network, a feature fusion network, and a detection head connected in sequence. In the process of inputting the fused image into the inventory model to obtain the target image, features can be first extracted from the fused image through the feature extraction network and a preset attention mechanism (such as a coordinate attention mechanism) to obtain a feature map. Therefore, a suitable position (such as after a convolution layer or a pooling layer) can be selected in the feature extraction network to embed a coordinate attention module (which can receive a feature map as input and output a feature map weighted by a coordinate attention mechanism) so as to perform weighted processing on the extracted features. Then, the output of the coordinate attention module (i.e., the feature map) can be used as the input of a subsequent network layer so as to utilize the weighted features in subsequent processing. It can be understood that the core of the coordinate attention module is to calculate the attention weights, which are used to weight each position in the input feature map, wherein the weights can be calculated based on the position information and the channel information. Thereafter, the feature map can be fused according to a preset attention mechanism (such as a spatial attention mechanism) and a feature fusion network to obtain the target image.
[0041] As another option of the embodiment of the present application, a target image is obtained according to the feature map, a preset attention mechanism and a feature fusion network, including: Calculate the attention weight value corresponding to the feature map according to the preset attention mechanism; The feature map is fused through the feature fusion network and the attention weight value to obtain the target image.
[0042] Specifically, in the process of fusing the feature map according to the spatial attention mechanism and the feature fusion network to obtain the target image, a module (i.e., spatial attention module) that can receive the feature map as input and output the spatial attention weight can be constructed through the convolution layer and the pooling layer. This module can use activation functions such as softmax to convert the extracted information into spatial attention weights (i.e., the attention weight value corresponding to the feature map). It is worth noting that the spatial attention mechanism focuses more on the spatial position of the image, that is, which areas or pixels are more critical to the current task (such as classification, detection, segmentation, etc.).
[0043] Next, a spatial attention module can be embedded in the appropriate position of the feature fusion network (such as before feature concatenation, addition or multiplication steps), and the output of the spatial attention module (i.e., the spatial attention weight) can be multiplied element-by-element with the original feature map to weight the spatial position in the feature map, thereby achieving feature fusion of the feature map through the feature fusion network and the attention weight value, and then using the weighted feature map as the input of the subsequent network layer, and obtaining the target image after calculation of the entire inventory model.
[0044] Step 104: Calculate the inventory result according to the material frame in the target image and the confidence of the material frame.
[0045] Specifically, after obtaining the target image, reliable material boxes (such as material boxes with a confidence level higher than 0.9) can be screened out according to the material boxes in the target image and the confidence levels of the material boxes and counted, thereby determining the material boxes belonging to the lineside warehouse materials, and determining the inventory results based on the counting results to achieve accurate inventory of the lineside warehouse materials, wherein the inventory results are the number of materials included in the initial image.
[0046] As another option of the embodiment of the present application, the inventory result is calculated according to the material frame in the target image and the confidence of the material frame, including: When the confidence level of the material box in the target image is higher than a preset third threshold, the material box is included in the inventory result; wherein the preset third threshold is a positive number less than 1, and the confidence level is the probability that the material box contains the material, which is calculated by the inventory model.
[0047] Specifically, in the process of calculating the inventory result based on the material box in the target image and the confidence of the material box, when the confidence of the material box in the target image is higher than the preset third threshold (such as 0.85), it means that the probability that the material box contains the lineside warehouse material is very high. Therefore, the material box can be included in the inventory result, and the statistical inventory result is obtained until all the material boxes in the target image are identified. It can be understood that since the confidence represents the probability that the model believes that the input sample belongs to a certain category, and in this solution, the confidence value is a value between 0 and 1, the preset third threshold is also between 0 and 1, that is, a positive number less than 1, and usually a positive number closer to 1 is used, such as 0.88, 0.92, 0.96, etc.
[0048] See also Figure 2 , Figure 2 A schematic structural diagram of a lineside warehouse material inventory device provided in an embodiment of the present application is shown.
[0049] like Figure 2 As shown, the lineside warehouse material inventory counting device may include at least a first processing module 201, a second processing module 202, a third processing module 203 and a fourth processing module 204, wherein: The first processing module 201 is used to obtain an initial image set of materials in the line warehouse; wherein the initial image set includes at least two initial images; The second processing module 202 is used to extract edge information of each initial image, and fuse the edge information of each initial image with the initial image to obtain a fused image; The third processing module 203 is used to construct an inventory model based on the initial image set, and input the fused image into the inventory model to obtain a target image; The fourth processing module 204 is used to calculate the inventory result according to the material frame in the target image and the confidence of the material frame; wherein the inventory result is the quantity of materials included in the initial image.
[0050] In some possible embodiments, extracting edge information of each initial image includes: The second processing module 202 is specifically used for: Calculating the gradient amplitude of each initial image in the horizontal direction by a preset operator to obtain a first gradient amplitude; Calculating the gradient amplitude of each initial image in the vertical direction by a preset operator to obtain a second gradient amplitude; The edge information of each initial image is obtained according to the first gradient amplitude and the second gradient amplitude.
[0051] In some possible embodiments, edge information of each initial image is fused with the initial image to obtain a fused image, including: The second processing module 202 is specifically used for: Calculate the pixel variance of each initial image; When it is detected that the pixel variance is greater than a preset first threshold, weighted fusion processing is performed on the edge information of the initial image corresponding to the pixel variance and the initial image to obtain a fused image.
[0052] In some possible embodiments, fusing the edge information of each initial image with the initial image to obtain a fused image further includes: The second processing module 202 is specifically used for: Calculate the contrast value of each initial image according to a preset formula; When it is detected that the contrast value is less than a preset second threshold, edge information of the initial image corresponding to the contrast value and the initial image are superimposed and fused to obtain a fused image.
[0053] In some possible embodiments, the inventory model includes a feature extraction network and a feature fusion network; Input the fused image into the inventory model to obtain the target image, including: The third processing module 203 is specifically used for: The fused image is input into the inventory model, and features are extracted from the fused image through the feature extraction network and the preset attention mechanism to obtain a feature map; According to the feature map, the preset attention mechanism and the feature fusion network, the target image is obtained.
[0054] In some possible embodiments, obtaining a target image according to a feature map, a preset attention mechanism, and a feature fusion network includes: The third processing module 203 is specifically used for: Calculate the attention weight value corresponding to the feature map according to the preset attention mechanism; The feature map is fused through the feature fusion network and the attention weight value to obtain the target image.
[0055] In some possible embodiments, calculating the inventory result according to the material box in the target image and the confidence of the material box includes: The fourth processing module 204 is specifically used for: When the confidence level of the material box in the target image is higher than a preset third threshold, the material box is included in the inventory result; wherein the preset third threshold is a positive number less than 1, and the confidence level is the probability that the material box contains the material, which is calculated by the inventory model.
[0056] See also Figure 3 , Figure 3 A structural schematic diagram of another line-side warehouse material inventory device provided in an embodiment of the present application is shown.
[0057] like Figure 3 As shown, the lineside warehouse material inventory counting device 300 may include at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 and at least one communication bus 302 .
[0058] The communication bus 302 may be used to realize the connection and communication among the above-mentioned components.
[0059] The user interface 303 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.
[0060] The network interface 304 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.
[0061] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect the various parts of the entire line-side warehouse material inventory device 300, and executes various functions and processes data of the line-side warehouse material inventory device 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 301 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, but may be implemented separately through a chip.
[0062] The memory 305 may include a RAM or a ROM. Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage device located away from the aforementioned processor 301. As Figure 3As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a line-side warehouse material inventory application.
[0063] Specifically, the processor 301 may be used to call the lineside warehouse material inventory application stored in the memory 305, and specifically perform the following operations: Acquire an initial image set of materials in the line side warehouse; wherein the initial image set includes at least two initial images; Extracting edge information of each initial image, and fusing the edge information of each initial image with the initial image to obtain a fused image; Building an inventory model based on the initial image set, and inputting the fused image into the inventory model to obtain the target image; The counting result is calculated according to the material frame in the target image and the confidence of the material frame; wherein the counting result is the number of materials included in the initial image.
[0064] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.
[0065] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0066] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0067] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0068] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0069] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.
[0071] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct related hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0072] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A method for counting materials in a line warehouse, characterized in that: include: Acquire an initial image set of materials in a line-side warehouse; wherein the initial image set includes at least two initial images; Extracting edge information of each of the initial images, and fusing the edge information of each of the initial images with the initial image to obtain a fused image; Building an inventory model based on the initial image set, and inputting the fused image into the inventory model to obtain a target image; An inventory result is calculated according to the material box in the target image and the confidence of the material box; wherein the inventory result is the quantity of materials included in the initial image.
2. The method according to claim 1, characterized in that The step of extracting edge information of each of the initial images comprises: Calculating the gradient amplitude of each of the initial images in the horizontal direction by using a preset operator to obtain a first gradient amplitude; Calculating the gradient amplitude of each of the initial images in the vertical direction by using the preset operator to obtain a second gradient amplitude; The edge information of each of the initial images is obtained according to the first gradient amplitude and the second gradient amplitude.
3. The method according to claim 1, characterized in that The step of fusing the edge information of each of the initial images with the initial image to obtain a fused image comprises: Calculating the pixel variance of each of the initial images; When it is detected that the pixel variance is greater than a preset first threshold, weighted fusion processing is performed on the edge information of the initial image corresponding to the pixel variance and the initial image to obtain a fused image.
4. The method according to claim 1, characterized in that: The step of fusing the edge information of each of the initial images with the initial image to obtain a fused image further includes: Calculating the contrast value of each of the initial images according to a preset formula; When it is detected that the contrast value is less than a preset second threshold, edge information of the initial image corresponding to the contrast value and the initial image are superimposed and fused to obtain a fused image.
5. The method according to claim 1, characterized in that The inventory model includes a feature extraction network and a feature fusion network; The step of inputting the fused image into the inventory model to obtain a target image comprises: Inputting the fused image into the inventory model, and extracting features from the fused image through the feature extraction network and the preset attention mechanism to obtain a feature map; A target image is obtained according to the feature map, the preset attention mechanism and the feature fusion network.
6. The method according to claim 5, characterized in that The step of obtaining a target image according to the feature map, the preset attention mechanism and the feature fusion network includes: Calculating an attention weight value corresponding to the feature map according to the preset attention mechanism; The feature fusion network and the attention weight value are used to perform feature fusion on the feature map to obtain a target image.
7. The method according to claim 1, characterized in that The calculating the inventory result according to the material frame in the target image and the confidence of the material frame includes: When the confidence level of identifying the material box in the target image is higher than a preset third threshold, the material box is included in the inventory result; wherein the preset third threshold is a positive number less than 1, and the confidence level is the probability that the material box contains the material, which is calculated by the inventory model.
8. A material inventory device for lineside warehouse, characterized in that: include: The first processing module is used to obtain an initial image set of materials in the line warehouse; wherein the initial image set includes at least two initial images; A second processing module is used to extract edge information of each of the initial images, and fuse the edge information of each of the initial images with the initial image to obtain a fused image; A third processing module is used to construct an inventory model based on the initial image set, and input the fused image into the inventory model to obtain a target image; The fourth processing module is used to calculate the inventory result according to the material box in the target image and the confidence of the material box; wherein the inventory result is the number of materials included in the initial image.
9. A material inventory device for lineside warehouse, characterized in that: including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.