Goods storage inventory identification system and method based on image identification

Abstract

The invention provides a cargo storage inventory recognition system and method based on image recognition, and belongs to the technical field of cargo storage management.The cargo storage inventory recognition system comprises the steps that firstly, storage area multi-source image sets which are provided with collection position identifiers and collected by different collection devices at different time periods are obtained, and then space correlation modeling is conducted on the storage area multi-source image sets; the method comprises the steps of obtaining an image space correlation model, identifying storage state features of stored goods based on the image space correlation model, including a stacking form and a goods allocation occupation feature, and generating a stored goods distribution model marked with image space correlation information according to the storage state features, and finally, according to the distribution model, generating an inventory list containing the cargo identifier and the storage position information, thereby efficiently and accurately completing the cargo storage inventory work.

Description

Technical Field

[0001] This invention relates to the field of cargo storage management technology, and more specifically, to a cargo storage inventory identification system and method based on image recognition. Background Technology

[0002] In the field of warehousing management, accurate and efficient inventory counting is a key link in ensuring inventory accuracy, optimizing warehousing resource allocation, and improving logistics operational efficiency. Traditional warehousing inventory counting methods mainly rely on manual operation. Staff need to carry inventory lists and check the identification information of goods against the actual storage situation in the storage area one by one. This not only consumes a lot of manpower and time, but is also prone to errors in inventory results due to human negligence, affecting the accuracy of inventory data.

[0003] With the development of technology, some inventory methods based on simple image acquisition have been gradually applied. However, these existing methods typically use a single acquisition device to collect images of stored goods at fixed intervals, resulting in relatively limited image information that cannot comprehensively reflect the actual storage status of the goods. Due to differences in the performance and shooting angles of different acquisition devices, as well as the influence of factors such as lighting and occupancy in the storage environment, images from a single source are difficult to accurately identify the stacking pattern of goods and the occupancy of storage locations. Furthermore, existing methods lack effective analysis and modeling of the spatial relationships between multi-source images, making it impossible to establish positional correspondences between different images. This makes it difficult to accurately associate the identification information and storage location information of goods when generating inventory lists, significantly reducing the accuracy and practicality of the inventory results. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a cargo storage inventory identification method based on image recognition, the method comprising: A multi-source image set of the storage area is obtained, wherein the multi-source image set contains images of stored goods collected by different acquisition devices at different time periods, and the images of stored goods have corresponding acquisition location identifiers; Spatial correlation modeling is performed on the multi-source image set to establish the positional correspondence between images of different stored goods, thus obtaining an image spatial correlation model; Based on the image spatial association model, the storage status characteristics of warehouse goods are identified, including the stacking pattern characteristics and storage space occupancy characteristics of the goods. A distribution model of the stored goods is generated based on the storage status characteristics, and the distribution model is marked with the image spatial association information corresponding to each goods; An inventory list of stored goods is generated based on the distribution model. The inventory list includes the identification information of the goods and their corresponding storage location information.

[0005] In another aspect, embodiments of the present invention also provide a cargo storage inventory identification system based on image recognition, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0006] Based on the above, this invention acquires a multi-source image set of a storage area, including images collected by different acquisition devices at different times and with acquisition location identifiers. Spatial correlation modeling is then performed on this multi-source image set to establish positional correspondences between images of different stored goods, forming an image spatial correlation model. This effectively solves the problem of unclear spatial relationships between multi-source images, enabling accurate association of goods information in different images. Based on this image spatial correlation model, the storage status characteristics of stored goods are identified, including stacking morphology and location occupancy characteristics, providing a deep and detailed understanding of the actual storage situation. A storage goods distribution model is generated based on the storage status characteristics, marked with the corresponding image spatial correlation information for each good. Finally, an inventory list containing goods identification information and corresponding storage location information is generated based on the distribution model, enabling accurate and rapid completion of goods inventory work, greatly improving the accuracy and efficiency of inventory counting. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the execution flow of the image recognition-based cargo storage and inventory identification method provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of exemplary hardware and software components of the image recognition-based cargo storage and inventory identification system provided in an embodiment of the present invention. Detailed Implementation

[0009] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a cargo storage inventory identification method based on image recognition, according to an embodiment of the present invention. The following is a detailed description of this cargo storage inventory identification method based on image recognition.

[0010] Step S110: Obtain a multi-source image set of the storage area. The multi-source image set contains images of stored goods collected by different acquisition devices at different times. The images of stored goods have corresponding acquisition location identifiers.

[0011] This embodiment uses a large e-commerce warehousing center as an application scenario. This warehousing center includes multiple shelving areas, stacking areas, and aisles, requiring regular inventory checks of various stored goods. To achieve comprehensive image acquisition, multiple image acquisition devices are deployed at different locations within the warehousing area.

[0012] Step S111: Collect initial warehouse images by image acquisition devices deployed at different locations in the warehouse area. The image acquisition devices include fixed cameras, cameras carried by mobile inspection robots, and aerial photography equipment carried by drones.

[0013] Inside this e-commerce warehousing center, fixed cameras are installed on the pillars between shelves, under the ceiling, and at aisle entrances. Each fixed camera is responsible for monitoring a specific shelf area or stacking area. For example, in area A, a fixed camera is installed on each side of the shelves, facing the front and side of the shelves respectively, to capture the placement of goods. Mobile inspection robots travel along pre-set aisle routes, and their onboard cameras can flexibly adjust their angles to capture images of goods on the middle and lower shelves, especially supplementing the coverage of corner areas that are difficult for fixed cameras to reach. Drones fly in the overhead space of the warehousing center, carrying aerial photography equipment to capture images of the distribution of goods throughout the warehousing area from an overhead perspective, especially capturing images of large stacks of goods in the stacking area. During operation, all these devices continuously collect initial warehousing images of their respective responsible areas, using a standardized image format to ensure compatibility for subsequent processing.

[0014] Step S112: Time-stamp the initial warehouse images, group the initial warehouse images acquired by the same acquisition device in consecutive time periods into the same image sequence, and each image sequence has a unique device identifier and time period code.

[0015] Once the initial warehouse images are captured, a capture time stamp can be immediately added to each image, accurate to the second. For example, an image captured by a fixed camera would have a specific year-month-day-hour-minute-second timestamp. Images captured by the same fixed camera within two consecutive hours can be grouped into the same image sequence. Each image sequence is assigned a unique device identifier and time period code. The device identifier consists of a combination of letters and numbers, such as FX-A-ZM for the front of area A of the fixed camera, YJ-05 for the mobile inspection robot, and WR-01 for the drone. The time period code consists of abbreviations of the start and end times. For example, an image sequence captured from 8:00 AM to 10:00 AM would have a time period code of 0800-1000. This allows for quick differentiation of image sequences captured by different devices at different times using the device identifier and time period code.

[0016] Step S113: Extract the acquisition location parameters of each initial warehouse image, and store the acquisition location parameters as acquisition location identifiers associated with the corresponding initial warehouse images in the image database. The acquisition location parameters are obtained based on the positioning module of the image acquisition device and include three-dimensional spatial coordinates and device orientation angle.

[0017] Image acquisition devices are all equipped with positioning modules. The positioning module of a fixed camera acquires the three-dimensional spatial coordinates of its installation position. For example, the three-dimensional spatial coordinates of a fixed camera in area A are (X1, Y1, Z1), where X1, Y1, and Z1 represent the X-axis, Y-axis, and Z-axis coordinates in the warehouse center coordinate system, respectively. The device's orientation angle is obtained through a direction sensor in the positioning module. For a fixed camera facing the shelf, its orientation angle is the horizontal angle θ1 and the vertical angle φ1. During the movement of the mobile inspection robot, the positioning module updates its position in real time. For each image acquired, it records the current three-dimensional spatial coordinates (X2, Y2, Z2) and orientation angle (horizontal angle θ2, vertical angle φ2). The drone's positioning module also records the three-dimensional spatial coordinates (X3, Y3, Z3) and orientation angle (horizontal angle θ3, vertical angle φ3) for each shot. These acquired position parameters are extracted and used as acquisition position identifiers, associated one-to-one with the corresponding initial warehouse images, and stored together in the image database. The storage uses an index, allowing for quick retrieval of the corresponding acquisition position identifier from the image, and vice versa.

[0018] Step S114: Detect overlapping areas in the initial warehouse images acquired by different image acquisition devices, identify warehouse goods image pairs with overlapping content by comparing edge features, and record the pixel ratio of the overlapping area between warehouse goods image pairs.

[0019] Initial warehouse images collected by different devices are retrieved from the image database. For example, an image captured by a fixed camera FX-A-ZM during the 08:00-10:00 time period and an image captured by a mobile inspection robot YJ-05 during the same time period are selected for overlapping region detection. First, the edge features of these two images are extracted, including the outline edges of goods and the edges of shelves. By comparing the edge features, it is determined whether there are identical goods or shelf parts in the two images. If so, the two images are identified as a pair of warehouse goods images with overlapping content. For the identified image pair, the number of pixels in the overlapping area is calculated, and then divided by the total number of pixels in each of the two images to obtain the proportion of overlapping pixels in each image. This proportion information and the location range of the overlapping area are recorded. For example, the coordinates of the overlapping area are the upper left corner (a1, b1) and lower right corner (c1, d1) in the first image, and the upper left corner (a2, b2) and lower right corner (c2, d2) in the second image.

[0020] Step S115: Integrate the initial warehouse images with acquisition location identifiers and the corresponding image sequences, warehouse cargo image pairs, and overlapping area pixel ratio information into a multi-source image set. The warehouse cargo images in the multi-source image set are arranged according to the acquisition time period and device identifier. Images acquired by different image acquisition devices in the same time period form an image group.

[0021] All initial warehouse images with acquisition location identifiers are integrated along with their corresponding image sequence information (device identifier, time period code), warehouse goods image pair information, and overlapping area pixel ratio information. During integration, images are sorted according to the chronological order of acquisition time, and images within the same time period are further categorized and arranged according to device identifiers. For example, images acquired by the fixed camera FX-A-ZM, the mobile inspection robot YJ-05, and the drone WR-01 during the 0800-1000 time period will form an image group containing all images acquired by these three devices during this time period, along with related sequences, image pairs, and overlap ratio information. This integration creates a multi-source image set, facilitating subsequent spatial correlation modeling.

[0022] Step S120: Perform spatial correlation modeling on the multi-source image set to establish the positional correspondence between images of different stored goods and obtain the image spatial correlation model.

[0023] After acquiring a set of images from multiple sources, spatial correlation modeling is required to clarify the positional relationships between different images.

[0024] Step S121: Extract edge feature points and texture feature points from each warehouse cargo image from the multi-source image set. Edge feature points are the intersection points of cargo outlines and backgrounds in the warehouse cargo images, and texture feature points are pixels with unique texture patterns on the cargo surface.

[0025] Step S1211: Preprocess the warehouse goods image by performing noise reduction and contrast enhancement in sequence, and then convert the preprocessed warehouse goods image to grayscale to convert the color warehouse goods image to a grayscale image.

[0026] First, a warehouse goods image is selected from a multi-source image set. Denoising is then performed using Gaussian filtering, where a suitable kernel size is set to smooth out noise and reduce graininess. Following denoising, contrast enhancement is applied using histogram equalization to adjust the grayscale distribution, making details clearer, such as making darker edges of goods more prominent. After preprocessing, the color image is converted to grayscale. The red, green, and blue channels of each pixel are calculated according to a set ratio to obtain a grayscale value, thus converting the color image to grayscale for subsequent feature point extraction.

[0027] Step S1212: Perform edge detection processing on the grayscale image, use a multi-scale edge detection operator to identify regions in the grayscale image where grayscale values ​​change abruptly at different scales, and generate a multi-scale edge response map.

[0028] Multi-scale edge detection operators, such as the multi-scale Canny operator, are applied to grayscale images. First, multiple different scale parameters are set, each corresponding to a different Gaussian kernel standard deviation. At each scale, a convolution operation is performed on the grayscale image to calculate the gradient value. Then, non-maximum suppression and double thresholding are applied to extract edge information at that scale. The edge information obtained from different scales is fused to generate a multi-scale edge response map. The value of each pixel in this multi-scale edge response map represents the edge response intensity at that location at different scales. Regions with abrupt changes in grayscale values ​​will show obvious peaks in the response map.

[0029] Step S1213: Extract continuous curve segments from the multi-scale edge response map, take the pixels on the curve segments that meet the feature requirements as candidate edge points, encode the candidate edge points using chain code, and retain the edge points that form the complete cargo outline.

[0030] In the multi-scale edge response map, continuous curve segments are formed by connecting adjacent high-response pixels. For each curve segment, feature requirements are set, such as the length and curvature of the curve segment. Pixels on curve segments that meet these requirements are selected as candidate edge points. Chain code encoding is performed on the candidate edge points. The chain code uses an 8-direction chain code, where each candidate edge point is represented by a direction value indicating its relative position to the previous point. Through chain code encoding, it can be determined whether the candidate edge points form a complete cargo outline. Edge points that can close to form a cargo outline are retained, while scattered edge points that cannot form a complete outline are removed, resulting in the final edge feature points.

[0031] Step S1214: Extract texture features from the grayscale image, use texture analysis algorithm to identify texture regions on the surface of the goods that have periodic or regular distribution, and extract representative pixels from the texture regions as candidate texture feature points.

[0032] A texture analysis algorithm based on gray-level co-occurrence matrix (GLCM) is used to process grayscale images. First, appropriate distance and angle parameters are set, and the GLCM of the grayscale image is calculated at different distances and angles. Texture feature parameters such as energy, entropy, and contrast are extracted from the GLCM. Based on these texture feature parameters, texture regions with periodic or regular distributions on the surface of goods are identified, such as stripes or grid patterns on packaging. Within these texture regions, representative pixels located at intersections and turning points of the texture patterns are selected as candidate texture feature points.

[0033] Step S1215: Filter the candidate texture feature points, remove the candidate texture feature points whose distribution density does not meet the set requirements, retain the feature points that can reflect the unique texture of the cargo surface, arrange the filtered edge feature points and texture feature points according to their coordinate order in the grayscale image, record the coordinate value of each feature point, the corresponding grayscale value change or texture description information, and determine the feature points as edge feature points and texture feature points of the warehouse cargo image.

[0034] The distribution density of candidate texture feature points is calculated by counting the number of candidate texture feature points per unit area. A threshold range for distribution density is set to remove candidate texture feature points that are too numerous (too dense) or too few (too sparse), retaining those feature points that are relatively evenly distributed and can reflect the unique texture of the goods surface. The filtered edge feature points and texture feature points are arranged according to their X and Y coordinates in the grayscale image. For each edge feature point, its coordinate value (x, y) and the grayscale value change at that point are recorded, such as the magnitude and direction of the grayscale value change; for each texture feature point, its coordinate value (x, y) and the corresponding texture description information, such as the direction and period of the texture, are recorded. The feature points processed in this way are the edge feature points and texture feature points of the warehouse goods image.

[0035] Step S122: Determine the matching of edge feature points between images of different stored goods by the relative positional relationship of feature points and the difference in gray values, and determine the matching of texture feature points by the similarity and distribution density of texture patterns.

[0036] Two images of warehouse goods, image A and image B, are selected from a multi-source image set. For edge feature point matching, the relative positional relationship between each edge feature point in image A and each edge feature point in image B is first calculated, i.e., the relative distance and relative angle are obtained through coordinate differences. Simultaneously, the grayscale value difference between the two edge feature points is calculated, i.e., the absolute difference in grayscale values. An allowable error range for the relative positional relationship and a threshold for the grayscale value difference are set. When the relative positional relationship of two edge feature points is within the allowable error range and the grayscale value difference is less than the threshold, the two edge feature points are considered to be likely to match. For texture feature point matching, the similarity of the texture patterns in the regions where texture feature points are located in images A and B is compared. This is achieved by calculating the similarity of texture feature parameters, and simultaneously comparing whether the distribution density of texture feature points is similar. When the texture pattern similarity meets the set standard and the distribution density is similar, the two texture feature points are considered to be likely to match.

[0037] Step S123: Based on the matching of edge feature points and texture feature points, select warehouse cargo image pairs that meet the set requirements for comprehensive matching degree, and mark the warehouse cargo image pairs as associated image pairs.

[0038] For images A and B, count the number of matched edge feature points and the number of matched texture feature points. Calculate the matching rate of edge feature points (the ratio of the number of matches to the total number of edge feature points in image A) and the matching rate of texture feature points (the ratio of the number of matches to the total number of texture feature points in image A), respectively. Define a method for calculating the overall matching degree, such as weighting and summing the edge feature point matching rates and texture feature point matching rates according to a set weight. When the overall matching degree is greater than a set threshold, select this pair of warehouse goods images, mark them as a related image pair, and record the matched feature point information.

[0039] Step S124: Perform coordinate transformation processing on the warehouse goods images in the associated image pair. Based on the three-dimensional spatial coordinates in the acquisition location identifier and the equipment orientation angle, transform the pixel coordinates of different images to a unified warehouse space coordinate system to generate the transformed normalized coordinates.

[0040] For images A and B in a paired image pair, their 3D spatial coordinates and device orientation angles are obtained from their acquisition location identifiers. Image A has 3D spatial coordinates (Xa, Ya, Za) and orientation angles of horizontal angle θa and vertical angle φa; image B has 3D spatial coordinates (Xb, Yb, Zb) and orientation angles of horizontal angle θb and vertical angle φb. Based on these parameters, a transformation relationship from pixel coordinates to a storage space coordinate system is established. This transformation involves the camera's intrinsic parameters (such as focal length and principal point coordinates) and extrinsic parameters (rotation matrix and translation vector). Through perspective projection transformation, the coordinates (u, v) of each pixel in images A and B are converted to coordinates (X, Y, Z) in the storage space coordinate system. After the transformation, normalized coordinates are obtained, ensuring that the coordinates of different images are in the same coordinate system, facilitating subsequent positional relationship analysis.

[0041] Step S125: Based on the transformed normalized coordinates, determine the overlap range, position offset, and angle deviation of the cargo area in the associated image pair. The position offset refers to the offset of the coordinates of the same cargo in different images, and the angle deviation refers to the deviation of the display angle of the same cargo in different images.

[0042] Under normalized coordinates, identify the cargo regions contained in images A and B of the associated image pair. By comparing the coordinate ranges of the two cargo regions, determine their overlap range, i.e., the minimum and maximum X-coordinates, minimum and maximum Y-coordinates, minimum and maximum Z-coordinates of the overlapping region. For the same cargo, the normalized coordinates in image A are (Xa1, Ya1, Za1), and the normalized coordinates in image B are (Xb1, Yb1, Zb1). Calculate the coordinate difference (Xb1-Xa1, Yb1-Ya1, Zb1-Za1) to represent the positional offset. Calculate the difference in the cargo's orientation angle between the two images to obtain the angular deviation. For example, if the cargo's orientation angle in image A is αa and in image B is αb, the angular deviation is αb-αa.

[0043] Step S126: Construct an image spatial association model in the form of a node association table to record the positional correspondence, overlap range, positional offset and angle deviation of each associated image pair. Here, nodes represent images of stored goods, and the node association table records the positional association parameters and feature point matching details between images.

[0044] Each image of stored goods is treated as a node, with the node identifier being a unique number for the image. For each pair of associated images, a record is created in the node association table. This record includes the identifiers of the two nodes, their positional correspondence (such as coordinate transformation parameters), the coordinate values ​​of the overlapping area, the coordinate difference for positional offsets, and the angle difference for angular deviations. Simultaneously, the node association table also records feature point matching details, including the coordinate information of matched edge feature point pairs and texture feature point pairs. Through this node association table, an image spatial association model is constructed, which clearly reflects the spatial association relationships between different images of stored goods.

[0045] Step S130: Identify the storage status characteristics of warehouse goods based on the image spatial association model. The storage status characteristics include the stacking pattern characteristics of the goods and the occupancy characteristics of the storage space.

[0046] By utilizing the constructed image spatial correlation model, the storage status of warehouse goods is further analyzed, thereby obtaining stacking morphology features and storage location occupancy features.

[0047] Step S131: Extract cargo area information from the associated image pairs from the image spatial association model, and combine the position correspondence, position offset and angle deviation in the image spatial association model to stitch together the display areas of the same cargo in different images to determine the complete display area of ​​the same cargo.

[0048] Information about associated image pairs is obtained from the node association table of the image spatial association model. For example, the cargo region information of the associated image pair C and D is extracted, including their cargo region coordinate range in normalized coordinates. Based on the coordinate transformation parameters in the positional correspondence, the cargo regions in images C and D are calibrated, considering positional offsets and angular deviations. The display areas of the same cargo in the two images are then stitched together. During stitching, the cargo region of one image is used as a reference, and the cargo region of the other image is adjusted according to offsets and deviations before being superimposed. Duplicate parts are removed to obtain a complete display area of ​​the same cargo, which contains all display information of the cargo at different angles and positions.

[0049] Step S132: Perform morphological analysis on the complete display area, identify the number of stacking layers, stacking density and stacking method of the goods through region segmentation and feature extraction, and combine the number of stacking layers, stacking density and stacking method into the stacking morphological features of the goods.

[0050] Step S1321: Based on the boundary coordinates of each sub-region in the sub-region set, extract the height information and contour information of each sub-region. The height information is obtained by combining the depth information of the image with the known size ratio of the goods and converted into the actual units of the warehouse space coordinate system. The contour information is extracted under normalized coordinates and kept consistent with the actual units to form a sub-region feature dataset.

[0051] In e-commerce warehousing scenarios, the complete display area of ​​the same product is divided into multiple sub-regions using a region segmentation algorithm. Each sub-region corresponds to an independent part of the product stack. For example, a stack of cardboard boxes is divided into multiple sub-regions, each representing one box. The boundary coordinates of each sub-region are obtained from the set of sub-regions, which are normalized coordinates from (Xmin, Ymin, Zmin) to (Xmax, Ymax, Zmax).

[0052] When extracting height information, the depth information of the image is first acquired using a depth sensor. The depth value of each pixel represents the distance from that point to the acquisition device. Combined with the known dimensions of the goods, such as the ratio between the actual height of a cardboard box and its height in the image, the depth information is converted into actual height units (e.g., meters) in the warehouse space coordinate system. For example, if the depth information of a certain sub-region in the image corresponds to a height of h pixels, this is converted to an actual height of H meters based on the proportional relationship.

[0053] Contour information extraction is performed in normalized coordinates, obtaining the coordinates of each vertex of the sub-region boundary. These coordinates are kept consistent with the actual units of the warehouse space coordinate system, forming the contour polygon of the sub-region. The height and contour information of each sub-region are then combined to form a sub-region feature dataset. Each sub-region corresponds to one record in the dataset, containing the height value and a list of contour vertex coordinates.

[0054] Step S1322: Based on the height information and relative positional relationship in the sub-region feature dataset, construct a spatial distribution model of the sub-region, determine the stacking order of goods through this spatial distribution model, determine the sub-region arrangement order and spatial distribution density from the bottom to the top, and generate a sub-region arrangement sequence.

[0055] By utilizing the height information in the sub-region feature dataset, the actual height values ​​of each sub-region are compared. Simultaneously, the relative positional relationships between sub-regions are considered; that is, the relative distance and orientation of each sub-region in the horizontal direction (XY plane) are calculated using the contour vertex coordinates. When constructing the spatial distribution model, each sub-region is treated as a spatial point with coordinates (X_center, Y_center, H), where X_center and Y_center are the center coordinates of the sub-region contour, and H represents the height information of the sub-region.

[0056] By analyzing the positional relationships of points in the spatial distribution model, the stacking order of goods is determined. For example, a sub-region with a lower height and no other sub-region below it is identified as the bottom layer, and a sub-region above the bottom layer that overlaps with it horizontally is identified as the upper layer. Following the order from bottom to top, the arrangement order of the sub-regions is determined, and the number of sub-regions per unit volume is calculated to obtain the spatial distribution density. After processing this information, a sub-region arrangement sequence is generated, where each element contains the sub-region's identifier, height information, and center coordinates.

[0057] Step S1323: Based on the sub-region arrangement sequence, count the number of sub-regions and use this number of sub-regions as the number of stacking layers of goods. At the same time, analyze the contact between adjacent sub-regions based on the contour information in the sub-region feature dataset. Determine the stacking tightness of goods by the ratio of the overlap length of the contact edge to the bottom edge length of the sub-region. Both the overlap length and the bottom edge length use coordinate values ​​with the same unit.

[0058] The number of sub-regions in the sub-region arrangement sequence is counted, and this number represents the number of stacking layers of the goods. For example, if the sub-region arrangement sequence contains 5 sub-regions, then the number of stacking layers is 5. For two adjacent sub-regions, their contour information is obtained from the sub-region feature dataset, and the edge segments of the two sub-regions on the contact surface are identified.

[0059] Calculate the overlap length of the contact edges, which is the length of the overlapping portion of the edge segments of the two sub-regions. Then calculate the total length of the bottom edge of the lower sub-region. Both calculations use the actual units of the storage space coordinate system (e.g., meters). Divide the overlap length by the bottom edge length. The larger the ratio, the tighter the stacking between adjacent sub-regions. For example, an overlap length of 0.8 meters and a bottom edge length of 1 meter result in a ratio of 0.8, indicating a high degree of stacking tightness.

[0060] Step S1324: Based on the relative position of each sub-region in the sub-region arrangement sequence and the contour information in the sub-region feature dataset, analyze the arrangement angle between sub-regions, determine the stacking pattern of the goods, and use this stacking pattern as the stacking method. The stacking pattern can be horizontal neat stacking, staggered stacking, tilted stacking, or vertical stacking.

[0061] From the sequence of sub-regions, adjacent sub-regions are selected, and their relative position vectors are calculated using their center coordinates. Combined with the edge directions in the contour information, the arrangement angle between the sub-regions is determined. If the line connecting the centers of adjacent sub-regions is parallel to the horizontal direction and their contour edges are aligned, with an arrangement angle of 0 degrees, it is determined to be horizontally neatly stacked. If adjacent sub-regions are staggered in the horizontal direction, and their arrangement angles alternate in a certain pattern, it is determined to be staggered stacking. If the line connecting the centers of sub-regions has a certain angle with the vertical direction and their contour edges are not parallel, it is determined to be tilted stacking. If the line connecting the centers of sub-regions is parallel to the vertical direction and their contour edges are aligned in the vertical direction, it is determined to be vertically stacked.

[0062] Step S1325: Combine the number of stacking layers, stacking density, and stacking method according to the preset coding rules to generate the stacking morphology features of the goods. The coding rules include hierarchical representation and combination of different stacking parameters so that different stacking morphology features have unique codes.

[0063] In the preset encoding rules, the number of stacking layers is represented by two digits, covering the maximum possible number of stacking layers; the stacking tightness is divided into several levels according to the ratio range, and each level is represented by a letter, such as A representing a ratio of 0.8-1.0, B representing 0.6-0.8, etc.; the stacking method is represented by a single digit, 1 representing horizontal neat stacking, 2 representing staggered stacking, 3 representing slanted stacking, and 4 representing vertical stacking.

[0064] The above parameters are combined in the order of "number of layers + density + method" to generate a unique code as the stacking morphology feature of the goods. For example, if the number of stacking layers is 5, the density ratio is 0.8 corresponding to A, and the stacking method is horizontal and neat stacking corresponding to 1, then the code is "05A1".

[0065] Step S133: Based on the normalized coordinates and device identifiers in the image spatial association model, determine the specific location of each item in the warehouse space coordinate system, associate it with the preset storage location division information of the warehouse, and determine the storage location area where each item is located. The storage location area is the preset storage area for goods in the warehouse.

[0066] Step S1331: Obtain the warehouse location division information, which includes the boundary coordinates of the location area, area identifier, area size and area type. The area type includes ordinary location, heavy-duty location and special location.

[0067] Retrieve preset storage location information from the warehouse management system. For example, in this e-commerce warehouse, the boundary coordinates of the storage location areas are (Xsmin, Ysmin, Zsmin) to (Xsmax, Ysmax, Zsmax) in the warehouse space coordinate system. Area identifiers consist of letters and numbers, such as A-001 being the first storage location in area A. Area dimensions are the length of the storage location area in the X, Y, and Z directions, such as 5 meters long, 3 meters wide, and 4 meters high. Area types are divided according to the characteristics of the stored goods: ordinary storage locations are used for standard-sized goods, heavy-duty storage locations are used for heavier goods, and special storage locations are used for perishable, fragile, or other special goods.

[0068] Step S1332: Convert the storage location division information to the same storage space coordinate system as the image space association model, and establish the mapping relationship between the storage location area and the storage space coordinates.

[0069] Since the location allocation information may be based on the original warehouse design coordinate system, it needs to be transformed to a warehouse space coordinate system consistent with the image space association model using coordinate transformation parameters. During the transformation process, it is essential to ensure that parameters such as the boundary coordinates and area dimensions of the location areas remain accurate in the new coordinate system. After establishing the mapping relationship, each location area corresponds to a specific spatial range in the warehouse space coordinate system, and the coordinate values ​​can be used to quickly determine which location area a particular position belongs to.

[0070] Step S1333: Extract the normalized coordinates of the complete display area of ​​each cargo from the image spatial association model, determine the minimum bounding rectangle of the cargo in the warehouse space coordinate system, and use the vertex coordinates of the minimum bounding rectangle as the position boundary of the cargo.

[0071] In the image spatial association model, the normalized coordinate range of the complete display area of ​​each item is found, and the smallest bounding rectangle that can completely contain the display area is determined by calculation. The vertex coordinates of the smallest bounding rectangle are represented in the storage space coordinate system, such as (Xmin_g, Ymin_g, Zmin_g), (Xmax_g, Ymin_g, Zmin_g), (Xmax_g, Ymax_g, Zmin_g), (Xmin_g, Ymax_g, Zmin_g), (Xmin_g, Ymin_g, Zmax_g), etc. These vertex coordinates together constitute the positional boundary of the item, which is used to represent the range occupied by the item in the storage space.

[0072] Step S1334: Compare the location boundary of the goods with the boundary coordinates of the storage area, calculate the area ratio of the goods in each storage area based on the location overlap, and then determine the storage area mainly included by the location boundary of the goods based on the size of the area ratio.

[0073] The smallest bounding rectangle of the cargo location boundary is compared with the boundary coordinates of each cargo location area to determine if there is any overlap. For cargo location areas that overlap, the proportion of the overlapping portion in the total area of ​​the cargo location boundary is calculated, i.e., the area percentage. For example, if the total area of ​​the cargo location boundary is S, and the overlapping area with cargo location area A is S1, then the area percentage is S1 / S. By comparing the area percentages of each cargo location area, the cargo location area with the largest percentage is initially determined as the area mainly containing the cargo.

[0074] Step S1335: When the location boundary of the goods spans multiple storage areas, the storage area with the largest area ratio of the goods in each storage area is determined as the primary storage area where the goods are located, and the secondary storage areas and their corresponding area ratios are recorded.

[0075] If the location boundary of goods overlaps with multiple storage areas simultaneously, such as spanning both storage area A and storage area B, calculate the area percentage of the goods in these two areas. Assuming the area percentage in storage area A is 60% and in storage area B is 30%, then storage area A is designated as the primary storage area, storage area B as the secondary storage area, and the 30% area percentage is recorded.

[0076] Step S1336: Associate the identified primary and secondary storage areas and their corresponding area percentages with the goods, record the storage area information where the goods are located, and establish a many-to-many relationship table between goods and storage areas.

[0077] The unique identifier of each item is associated with and stored in a primary storage location area, secondary storage location area, and their corresponding area percentages, forming a relationship table. Each record in the table contains the item identifier, a list of primary and secondary storage location area identifiers, and the area percentage of each secondary area. This relationship table allows for a clear lookup of the storage location for each item.

[0078] Step S134: Detect the area occupied by goods in the storage location area, calculate the volume ratio of the storage location based on the number of stacked layers of goods, and then determine the degree of occupancy and space utilization of the storage location by combining the actual size parameters of the storage location area. The degree of occupancy is expressed as the proportion of storage location space occupied.

[0079] The extent to which the goods occupy the storage space is determined by the coordinates of the goods' location boundaries and the storage area boundaries; that is, the volume of the goods within the storage area. The total volume of the goods is calculated based on the number of stacked layers and the average volume of each layer. The storage space occupancy ratio is the ratio of the total volume of the goods to the total volume of the storage area (calculated based on actual dimensions).

[0080] For example, if the total volume of the storage area is V and the total volume of the goods is Vg, then the volume ratio is Vg / V. The occupancy level is this volume ratio, and the space utilization is described according to the occupancy level, such as high, medium, or low, while also considering factors such as whether the goods are evenly distributed within the storage area.

[0081] Step S135: Associate the occupancy level of the storage location, the space utilization, and the location information of the storage location area as storage location occupancy features, integrate the stacking pattern features and storage location occupancy features as storage status features of the stored goods, and establish the association between storage status features and goods identification.

[0082] The occupancy rate (volume ratio), space utilization, and location information (boundary coordinates, area identifier) ​​of the storage location are combined to form the storage location occupancy characteristic. Then, the stacking characteristics of the goods (such as the code "05A1") are integrated with the storage location occupancy characteristic to form the storage status characteristic of the stored goods. Finally, the storage status characteristic is associated with the unique identifier of the goods to ensure that each item has a corresponding storage status characteristic record.

[0083] Step S140: Generate a distribution model of the stored goods based on the storage status characteristics. The distribution model is marked with the image spatial association information corresponding to each goods.

[0084] After obtaining the storage status characteristics of the stored goods, a distribution model is built based on this to intuitively display the distribution of goods in the storage area.

[0085] Step S141: Construct the basic framework of the distribution model based on the warehouse space coordinate system. The basic framework includes the physical boundaries of the warehouse area, aisle locations, shelf distribution, and storage location area division. Scale the model to the visual size according to a preset ratio.

[0086] Using the warehouse space coordinate system as a reference, information such as the physical boundaries of the warehouse area (e.g., wall locations), the coordinate range of aisles, the distribution of shelves, and the division of storage areas are integrated to construct the basic framework of the distribution model. The actual size is scaled up to a size suitable for visualization according to a preset ratio (e.g., 1:100), so that the basic framework can clearly present the overall layout of the warehouse area.

[0087] Step S142: Map the storage location occupancy feature in the storage status feature to the basic framework, mark the existence status, occupancy degree and space utilization of the goods in the corresponding storage location area, and use different filling modes to distinguish different occupancy degrees.

[0088] For example, step S1421: Analyze the location information, occupancy level, space utilization and storage area type of the storage area in the storage location occupancy characteristics, and determine the storage area to be marked and the corresponding marking parameters.

[0089] The occupancy characteristics of the storage locations are analyzed to extract their location information (boundary coordinates, area identifier), occupancy level (volume ratio), space utilization description, and area type. Based on this information, the storage locations that need to be marked within the basic framework are determined, and corresponding marking parameters are set, such as the type of fill mode and the content of the marking.

[0090] Step S1422: Set the fill mode of the marker according to the occupancy level classification standard. The fill mode includes fill color, fill density and fill pattern. The higher the occupancy level, the darker the fill color and the greater the fill density.

[0091] The system pre-defines occupancy levels, such as 0-30% for low occupancy, 30%-70% for medium occupancy, and 70%-100% for high occupancy. Each level is assigned a corresponding fill pattern; for example, low occupancy uses a light green, low-density dot pattern, medium occupancy uses a yellow, medium-density grid pattern, and high occupancy uses a dark red, high-density diagonal line pattern. These settings allow for a clear visual distinction of occupancy levels in different storage areas.

[0092] Step S1423: Locate the coordinate range corresponding to the location information of the storage area in the basic framework of the distribution model, and draw a basic marker box within the coordinate range according to the shape of the storage area. The size of the basic marker box is proportional to the actual size of the storage area.

[0093] Based on the location information of the storage area, find the corresponding coordinate range in the basic framework, and draw a basic marker frame according to the actual shape of the storage area (such as a rectangle or polygon). The size of the basic marker frame is determined according to the scaling ratio of the basic framework and is proportional to the actual size of the storage area to ensure that the marker frame can accurately correspond to the position and size of the storage area in the warehouse space.

[0094] Step S1424: Fill the basic mark box with a fill pattern corresponding to the occupancy level, and mark the relevant description of space utilization at the specified position of the basic mark box.

[0095] Within the drawn basic marking frame, fill with a pattern determined by the occupancy level, such as filling highly occupied storage areas with dark red, high-density diagonal lines. Simultaneously, above or beside the basic marking frame, label the space utilization status with descriptions such as "evenly distributed" or "locally congested."

[0096] Step S1425: When there are multiple goods in the same storage area, adjust the filling layout in the basic mark box according to the stacking shape and distribution of the goods, and use a layered filling method to represent the occupied area of ​​different goods.

[0097] If there are multiple goods in the same storage location area, the fill layout within the basic marker box should be adjusted based on the stacking characteristics of each good (such as the number of stacking layers and stacking method) and their distribution within the storage location area. For example, different colors can be used for layered fill, with each layer corresponding to one good or a group of good. The color can be used to distinguish the occupied area of ​​different good, while maintaining the overall fill pattern in line with the occupancy level of the storage location area.

[0098] Step S1426: After marking, check whether the correspondence between the filling mode and the degree of occupancy is correct, whether the labeling of space utilization is accurate, and whether the marking box completely includes the location boundary of the goods. Correct any errors found.

[0099] Conduct a comprehensive inspection of the marked storage areas to verify that the fill pattern matches the occupancy level, that the space utilization labeling accurately reflects the actual situation, and that the marking frame completely encompasses the location boundaries of the goods. If any issues are found, such as incorrect fill pattern, inaccurate labeling, or deviations in the marking frame range, correct them promptly to ensure the accuracy of the marking information.

[0100] Step S143: Extract the stacking morphology features from the storage status features, and mark the stacking morphology features in the form of a combination of graphic symbols in the corresponding storage location area. The shape of the graphic symbol corresponds to the stacking method, the size of the graphic symbol corresponds to the stacking tightness, and the number of graphic symbols corresponds to the number of stacking layers.

[0101] Based on the stacking pattern, select the corresponding graphic symbol shape. For example, use a rectangle for horizontal, neat stacks, a staggered rectangle for interlaced stacks, a parallelogram for angled stacks, and a vertical line for vertical stacks. The size of the graphic symbol is determined by the density of the stacking; the denser the stack, the larger the symbol. The number of graphic symbols corresponds to the number of stacking layers. For example, if there are 5 stacking layers, draw 5 corresponding graphic symbols in the storage area. Combine these graphic symbols and label them in the corresponding storage area to visually represent the stacking pattern of the goods.

[0102] Step S144: Extract the image spatial association information corresponding to each item from the image spatial association model, and add the image spatial association information as attribute labels to the corresponding item's marker in the distribution model. The attribute labels can be shown or hidden through interactive operations. The image spatial association information includes associated image identifiers, position offset, angle deviation, and feature point matching details.

[0103] From the node association table of the image spatial association model, extract the associated image identifiers (e.g., image numbers of associated image pairs), positional offsets (coordinate differences), angular deviations (angle differences), and feature point matching details (coordinate pairs of matched feature points) for each item. Organize this information into attribute labels and associate them with the corresponding item markers (e.g., graphic symbols or basic bounding boxes) in the distribution model. Interactive operations such as mouse clicks can show or hide these attribute labels for easy viewing of detailed association information.

[0104] Step S145: Integrate and optimize the labels in the distribution model, merge duplicate labels of the same goods in different associated images, and generate a distribution model of warehouse goods.

[0105] In the distribution model, the same item may have multiple tags due to being captured in different associated images. Although these tags all point to the same item, they may differ in location and display details. By comparing the item identification information, location boundaries, and image spatial association information associated with each tag, it is determined which tags belong to the same item. For duplicate tags identified as belonging to the same item, the tag with the most complete information and the most accurate location is retained as the primary tag. Useful information from other tags (such as feature descriptions from different angles) is integrated into the attribute label of the primary tag, and then duplicate tags are deleted. For example, a box of electronic products may be tagged in images captured by both a fixed camera and a mobile inspection robot. After comparing their unique codes and location boundaries to determine that they belong to the same item, the tag generated by the fixed camera image is retained as the primary tag. The angular deviation information of the item in the mobile inspection robot image is added to the attribute label of the primary tag, and then the tag corresponding to the mobile inspection robot is deleted. Through the above integration and optimization, it is ensured that each item in the distribution model has only one clear tag that contains all the key information of the item in different associated images, ultimately generating a complete and accurate warehouse item distribution model.

[0106] Step S150: Generate an inventory list of stored goods based on the distribution model. The inventory list includes the identification information of the goods and their corresponding storage location information.

[0107] After determining the distribution model of the stored goods, relevant information can be extracted based on the distribution model to generate a list that meets the inventory requirements.

[0108] Step S151: Perform a traversal scan of the cargo tags in the distribution model, and obtain the cargo identification information corresponding to each cargo tag by identifying the attribute labels associated with the cargo tags. The cargo identification information includes the cargo name, specifications and unique code. Extract and verify the cargo identification information from the warehouse cargo images associated with the image spatial association information.

[0109] Following the arrangement order of the storage locations in the distribution model, all cargo tags are scanned traversally. During the scan, the attribute tags associated with each cargo tag are read, and cargo identification information is extracted from them. For example, the attribute tag of a cargo tag records the cargo name as "wireless Bluetooth headset," the specification model as "BH-001," and the unique code as "WL-BH-001-20231005." To ensure the accuracy of the cargo identification information, the corresponding warehouse cargo image is found from the image spatial association information. Image recognition technology is used to extract the name, specification model, and code from the cargo packaging, and this information is compared and verified with the information in the attribute tags. If they match, the cargo identification information is confirmed to be correct; if there are discrepancies, the information extracted from the image is used for correction, and the content in the attribute tags is updated.

[0110] Step S152: Extract the location area identifier, primary location area and secondary location area information corresponding to each cargo mark in the distribution model, and combine the location division information to determine the specific storage location description of the location area. The storage location description includes the area number, shelf number, layer number and specific location.

[0111] For each cargo tag, its corresponding storage location area identifier is obtained from the distribution model, such as "HW-01-A-03", while identifying the primary and secondary storage location areas (if any). Combining this with the pre-defined storage location division information, the specific storage location corresponding to the storage location area identifier is analyzed. The area number corresponds to the large area division of the storage center, such as "HW-01" representing area A; the shelf number corresponds to the shelf within that area, such as "A" representing shelf number 1 in area A; the shelf level number corresponds to the shelf level, such as "03" representing the 3rd level; the specific location is determined based on the location of the cargo on the shelf, such as "left," "right," or "middle." For example, the storage location description corresponding to the storage location area identifier "HW-01-A-03" is "Area number HW-01, shelf number A, shelf level number 03, specific location left."

[0112] Step S153: Associate and match the cargo identification information with the corresponding storage location description, occupancy level, and stacking form characteristics to construct a target association table so that each cargo identification information corresponds to a unique association record, and the association record contains all relevant cargo location area information.

[0113] Each item's identification information (item name, specifications, unique code) is linked one-to-one with its corresponding storage location description (area number, shelf number, layer number, specific location), storage location occupancy (e.g., 70%), and stacking characteristics (e.g., 3 stacking layers, high compactness, horizontal and neat stacking). This linked information is compiled into a target association table, with each item's unique code serving as the primary key, ensuring that each item's identification information corresponds to a unique associated record in the table. The association record details all relevant storage location area information, including primary and secondary storage location areas (if any) and their area percentages. For example, in addition to the primary storage location area information, the association record for a certain item might also record the secondary storage location area "HW-01-A-04" and its area percentage of 10%.

[0114] Step S154: Sort the associated records in the target association table according to the spatial distribution order of the storage area to generate a preliminary inventory list. The sorting order follows the spatial order from left to right, from top to bottom, and from outside to inside.

[0115] Based on the actual distribution of storage locations within the warehouse space, a sorting rule is determined. First, the locations are arranged from left to right according to their area numbers. Within the same area, they are arranged from left to right according to their shelf numbers. Within the same shelf, they are arranged from top to bottom according to their level numbers. Within the same level, they are arranged from outside to inside. The associated records in the target association table are then sorted according to this rule. For example, goods in area number "HW-01" are sorted first, followed by goods in area number "HW-02"; within area "HW-01", goods in shelf number "A" are sorted first, followed by goods in shelf number "B"; within shelf "A", goods in level number "01" are sorted first, followed by goods in levels "02" and "03", and so on. After sorting, a preliminary inventory list is generated, listing the associated record information for each item in sequence.

[0116] Step S155: After performing multi-dimensional verification and correction on the preliminary inventory list, a final warehouse inventory list containing detailed information on goods and storage locations is generated.

[0117] The preliminary inventory list was verified from multiple dimensions. First, the identification information for each item was checked against the storage location description to ensure there were no mismatches. Second, the records of storage space occupancy and stacking characteristics were checked against the information in the distribution model. Third, the sorting was verified to conform to the spatial distribution order and to check for any errors. Finally, the total quantity of goods was counted and compared with the actual warehousing situation to check for omissions or duplicates. Any issues discovered during verification, such as incorrect storage location descriptions or reversed sorting, were promptly corrected. After corrections, the final inventory list was generated, clearly listing detailed information for each item (name, specifications, unique code) and storage location details (area number, shelf number, layer number, specific location, etc.), as well as storage space occupancy and stacking characteristics.

[0118] Figure 2 The illustration shows exemplary hardware and software components of an image recognition-based warehouse inventory identification system 100 that can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the image recognition-based warehouse inventory identification system 100 and to perform the functions in this application.

[0119] The image recognition-based cargo inventory identification system 100 can be a general-purpose server or a special-purpose server; both can be used to implement the image recognition-based cargo inventory identification method of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0120] For example, the image recognition-based warehouse inventory identification system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the image recognition-based warehouse inventory identification system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The image recognition-based warehouse inventory identification system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0121] For ease of explanation, only one processor is described in the image recognition-based warehouse inventory identification system 100. However, it should be noted that the image recognition-based warehouse inventory identification system 100 of this application may also include multiple processors. Therefore, the steps performed by one processor as described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the image recognition-based warehouse inventory identification system 100 performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0122] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-described image recognition-based cargo storage inventory identification method is implemented.

[0123] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for identifying and counting goods in a warehouse based on image recognition, characterized in that, The method includes: A multi-source image set of the storage area is obtained, wherein the multi-source image set contains images of stored goods collected by different acquisition devices at different time periods, and the images of stored goods have corresponding acquisition location identifiers; Spatial correlation modeling is performed on the multi-source image set to establish the positional correspondence between images of different stored goods, thus obtaining an image spatial correlation model; Based on the image spatial association model, the storage status characteristics of warehouse goods are identified, including the stacking pattern characteristics and storage space occupancy characteristics of the goods. A distribution model of the stored goods is generated based on the storage status characteristics, and the distribution model is marked with the image spatial association information corresponding to each goods; An inventory list of stored goods is generated based on the distribution model. The inventory list includes the identification information of the goods and their corresponding storage location information.

2. The image recognition-based cargo inventory identification method according to claim 1, characterized in that, The acquisition of the multi-source image set of the warehouse area includes: Initial warehouse images are acquired by image acquisition devices deployed at different locations in the warehouse area. The image acquisition devices include fixed cameras, cameras mounted on mobile inspection robots, and aerial photography equipment carried by drones. The initial warehouse images are time-stamped, and initial warehouse images acquired by the same acquisition device in consecutive time periods are grouped into the same image sequence. Each image sequence has a unique device identifier and time period code. Extract the acquisition location parameters of each initial warehouse image, and store the acquisition location parameters as acquisition location identifiers associated with the corresponding initial warehouse images in the image database. The acquisition location parameters are obtained based on the positioning module of the image acquisition device and include three-dimensional spatial coordinates and device orientation angle. Overlapping regions are detected in the initial warehouse images acquired by different image acquisition devices. By comparing edge features, warehouse goods image pairs with overlapping content are identified, and the pixel ratio of the overlapping region between the warehouse goods image pairs is recorded. The initial warehouse images with acquisition location identifiers and their corresponding image sequences, warehouse cargo image pairs, and overlapping area pixel ratio information are integrated into a multi-source image set. The warehouse cargo images in the multi-source image set are arranged according to the acquisition time period and device identifier. Images acquired by different image acquisition devices in the same time period form an image group.

3. The image recognition-based cargo inventory identification method according to claim 1, characterized in that, The step of performing spatial correlation modeling on the multi-source image set to establish the positional correspondence between images of different stored goods, and obtaining an image spatial correlation model, includes: Edge feature points and texture feature points of each warehouse cargo image are extracted from the multi-source image set. The edge feature points are the intersection points of the cargo outline and the background in the warehouse cargo image, and the texture feature points are the pixels with unique texture patterns on the cargo surface. The matching of edge feature points between images of different stored goods is determined by the relative positional relationship of feature points and the difference in gray values, and the matching of texture feature points is determined by the similarity and distribution density of texture patterns. Based on the matching results of edge feature points and texture feature points, warehouse cargo image pairs that meet the set requirements in terms of overall matching degree are selected, and the warehouse cargo image pairs are marked as associated image pairs; Coordinate transformation is performed on the images of stored goods in the associated image pairs. Based on the three-dimensional spatial coordinates in the acquisition location identifier and the equipment orientation angle, the pixel coordinates of different images are transformed to a unified storage space coordinate system to generate the transformed normalized coordinates. Based on the transformed normalized coordinates, the overlap range, position offset, and angle deviation of the cargo area in the associated image are determined. The position offset refers to the offset of the coordinates of the same cargo in different images, and the angle deviation refers to the deviation of the display angle of the same cargo in different images. An image spatial association model is constructed in the form of a node association table, which records the positional correspondence, overlap range, positional offset, and angular deviation of each associated image pair. The nodes represent images of stored goods, and the node association table records the positional association parameters and feature point matching details between images.

4. The image recognition-based cargo inventory identification method according to claim 3, characterized in that, The step of extracting edge feature points and texture feature points from each warehouse cargo image in the multi-source image set includes: The images of stored goods are preprocessed by performing noise reduction and contrast enhancement in sequence. The preprocessed images of stored goods are then converted to grayscale to transform the color images of stored goods into grayscale images. Edge detection processing is performed on the grayscale image, and a multi-scale edge detection operator is used to identify regions in the grayscale image where grayscale values ​​change abruptly at different scales, generating a multi-scale edge response map; Continuous curve segments are extracted from the multi-scale edge response map. Pixels on the curve segments that meet the feature requirements are used as candidate edge points. Chain code encoding is performed on the candidate edge points to retain the edge points that form the complete cargo outline. Texture features are extracted from grayscale images, and texture analysis algorithms are used to identify texture regions on the surface of goods that have periodic or regular distributions. Representative pixels are extracted from the texture regions as candidate texture feature points. Candidate texture feature points are filtered out, and those whose distribution density does not meet the set requirements are removed. Feature points that can reflect the unique texture of the cargo surface are retained. The filtered edge feature points and texture feature points are arranged according to their coordinate order in the grayscale image. The coordinate value of each feature point, the corresponding grayscale value change or texture description information are recorded. The feature points are determined as edge feature points and texture feature points of the warehouse cargo image.

5. The image recognition-based cargo inventory identification method according to claim 1, characterized in that, The identification of storage status features of warehouse goods based on the image spatial association model includes: The cargo area information in the associated image pairs is extracted from the image spatial association model. Combined with the position correspondence, position offset and angle deviation in the image spatial association model, the display area of ​​the same cargo in different images is stitched together to determine the complete display area of ​​the same cargo. The complete display area is subjected to morphological analysis. The number of stacking layers, the degree of stacking density, and the stacking method of the goods are identified through region segmentation and feature extraction. The number of stacking layers, the degree of stacking density, and the stacking method are combined into the stacking morphological features of the goods. Based on the normalized coordinates and device identifiers in the image spatial association model, the specific location of each item in the warehouse spatial coordinate system is determined. The warehouse's preset storage location information is then associated to determine the storage location area where each item is located. The storage location area is a preset storage area for goods in the warehouse. The area occupied by goods in the storage location is detected, and the volume ratio of the storage location is calculated based on the number of stacked layers of goods. Then, the degree of occupancy and space utilization of the storage location are determined by combining the actual size parameters of the storage location area. The degree of occupancy is expressed as the proportion of storage location space occupied. The occupancy level, space utilization, and location information of the storage area are associated as storage location occupancy features. Stacking pattern features and storage location occupancy features are integrated into storage status features of warehouse goods, and the association between storage status features and goods identification is established.

6. The image recognition-based cargo inventory identification method according to claim 5, characterized in that, The morphological analysis of the complete display area, through region segmentation and feature extraction, identifies the number of stacking layers, stacking density, and stacking method of the goods, and combines the number of stacking layers, stacking density, and stacking method into the stacking morphological features of the goods, including: Based on the boundary coordinates of each sub-region in the sub-region set, the height information and contour information of each sub-region are extracted. The height information is obtained based on the depth information of the image and combined with the known size ratio of the goods and converted into actual units in the warehouse space coordinate system. The contour information is extracted under normalized coordinates and kept consistent with the actual units to form a sub-region feature dataset. Based on the height information and relative positional relationships in the sub-region feature dataset, a spatial distribution model of the sub-region is constructed. The stacking order of goods is determined by this spatial distribution model, and the arrangement order and spatial distribution density of the sub-regions from the bottom to the top are determined to generate a sub-region arrangement sequence. Based on the number of sub-regions arranged according to the sub-region arrangement sequence, the number of sub-regions is used as the number of stacking layers of goods. At the same time, the contact between adjacent sub-regions is analyzed based on the contour information in the sub-region feature dataset. The stacking tightness of goods is judged by the ratio of the overlap length of the contact edge to the bottom edge length of the sub-region. Both the overlap length and the bottom edge length use coordinate values ​​with the same unit. Based on the relative position of each sub-region in the sub-region arrangement sequence and the contour information in the sub-region feature dataset, the arrangement angle between the sub-regions is analyzed to determine the stacking pattern of the goods. This stacking pattern is used as the stacking method, which can be horizontal neat stacking, staggered stacking, inclined stacking, or vertical stacking. The number of stacking layers, stacking density, and stacking method are combined according to a preset coding rule to generate the stacking morphology features of the goods. The coding rule includes a hierarchical representation and combination method for different stacking parameters so that different stacking morphology features have a unique code.

7. The image recognition-based cargo inventory identification method according to claim 5, characterized in that, The process involves determining the specific location of each item in the warehouse spatial coordinate system based on the normalized coordinates and device identifiers in the image spatial association model, associating this with the pre-defined warehouse location division information, and determining the location area for each item, including: Obtain warehouse storage location division information, which includes the boundary coordinates, area identifier, area size, and area type of the storage location area. The area type includes ordinary storage location, heavy storage location, and special storage location. The location division information is converted to the same storage space coordinate system as the image space association model to establish the mapping relationship between the location area and the storage space coordinates; The normalized coordinates of the complete display area of ​​each cargo are extracted from the image spatial association model, the minimum bounding rectangle of the cargo in the warehouse space coordinate system is determined, and the vertex coordinates of the minimum bounding rectangle are used as the position boundary of the cargo. The location boundary of the goods is compared with the boundary coordinates of the storage area. The area ratio of the goods in each storage area is calculated by the overlap of the locations. Then, the storage area mainly included by the location boundary of the goods is determined according to the size of the area ratio. When the location boundary of goods spans multiple storage areas, the storage area with the largest area proportion of goods in each storage area is determined as the primary storage area where the goods are located, and the secondary storage areas and their corresponding area proportions are recorded at the same time. Associate the identification of the primary and secondary storage areas, their corresponding area percentages, with the goods, record the storage area information where the goods are located, and establish a many-to-many relationship table between goods and storage areas.

8. The image recognition-based cargo inventory identification method according to claim 1, characterized in that, The step of generating a distribution model for stored goods based on the storage status characteristics includes: A basic framework for constructing a distribution model is built based on a warehouse space coordinate system. The basic framework includes the physical boundaries of the warehouse area, aisle locations, shelf distribution, and storage location area division, and is scaled to a visual size according to a preset ratio. Map the storage location occupancy features in the storage status features to the basic framework, mark the existence status, occupancy level and space utilization of goods in the corresponding storage location area, and use different filling modes to distinguish different occupancy levels; Extract the stacking pattern features from the storage status features, and mark the stacking pattern features in the form of a combination of graphic symbols in the corresponding storage location area. The shape of the graphic symbol corresponds to the stacking method, the size of the graphic symbol corresponds to the stacking tightness, and the number of graphic symbols corresponds to the number of stacking layers. Image spatial association information corresponding to each item is extracted from the image spatial association model. The image spatial association information is added to the corresponding item's marker in the distribution model in the form of attribute labels. The attribute labels are shown or hidden through interactive operations. The image spatial association information includes associated image identifiers, position offset, angle deviation, and feature point matching details. The labels in the distribution model are integrated and optimized by merging duplicate labels of the same goods in different associated images to generate a distribution model of warehouse goods.

9. The image recognition-based cargo inventory identification method according to claim 1, characterized in that, The step of generating an inventory list of stored goods based on the distribution model includes: The cargo tags in the distribution model are traversed and scanned. The cargo identification information corresponding to each cargo tag is obtained by identifying the attribute tags associated with the cargo tags. The cargo identification information includes cargo name, specifications and unique code. The cargo identification information is extracted and verified from the warehouse cargo images associated with image spatial association information. Extract the location area identifier, primary location area, and secondary location area information corresponding to each cargo tag in the distribution model. Combine the location division information to determine the specific storage location description of the location area. The storage location description includes the area number, shelf number, layer number, and specific orientation. The identification information of goods is associated with and matched with the corresponding storage location description, occupancy level and stacking form characteristics to construct a target association table so that each piece of goods identification information corresponds to a unique association record, and the association record contains all relevant storage location area information; Sort the associated records in the target association table according to the spatial distribution order of the storage areas to generate a preliminary inventory list. The sorting order follows the spatial order from left to right, from top to bottom, and from outside to inside. After multi-dimensional verification and correction of the initial inventory list, a final warehouse inventory list containing detailed information on goods and their storage locations is generated.

10. A cargo storage and inventory identification system based on image recognition, characterized in that, The device includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the image recognition-based cargo storage inventory identification method according to any one of claims 1-9.

Images