Efficient article sorting method and system of intelligent warehousing system

Abstract

The invention provides an efficient article sorting method and system of an intelligent warehousing system, and relates to the technical field of intelligent warehousing systems.The method comprises the steps that warehousing real-time environment data and a target image set for target objects to be sorted are obtained; inputting the target image set into a preset object classification model to determine a target classification corresponding to the target object; determining at least one candidate free goods allocation matched with the target classification, and determining a target transportation path of the target object according to each candidate free goods allocation, the storage real-time environment data and the target classification; determining a target clamping strategy matched with the target classification; and the target path and the target clamping strategy are sent to the AGV and the mechanical arm correspondingly, so that the AGV and the mechanical arm cooperatively sort the target objects. Therefore, the intelligent level of the intelligent warehousing system is remarkably improved, and automatic sorting tasks for various objects can be completed more intelligently.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent warehousing systems, and in particular to an efficient article sorting method and system for intelligent warehousing systems. Background Art

[0002] At present, intelligent warehousing systems generally use automatic sorting systems based on barcode or radio frequency identification (RFID) technology. These systems can efficiently process standard commodities that have been pre-marked, but their accuracy is greatly reduced for unmarked, irregularly shaped or newly launched commodities, and manual intervention is often required, resulting in low processing efficiency. In addition, traditional automatic sorting systems also have certain restrictions on the size, weight and material of items, and cannot fully meet the needs of diversified commodity processing. The system often requires a lot of manual intervention, which not only increases labor costs, but also reduces overall operating efficiency.

[0003] The industry has not yet proposed a better solution to the above problems. Summary of the invention

[0004] The present application provides an efficient article sorting method and system for an intelligent warehousing system, which is used to at least solve the problem that the traditional automatic sorting system relies heavily on commodity standardization, resulting in increased manual intervention costs and low operating efficiency.

[0005] The present application provides an efficient article sorting method for an intelligent warehousing system, comprising: obtaining real-time warehousing environment data and a target image set for a target object to be sorted; the real-time warehousing environment data is environmental data in the warehouse collected in real time through a sensor network, and the real-time warehousing environment data includes shelf layout information and obstacle distribution information; the target image set includes images of the target object captured from multiple angles; the sensor network includes a laser scanner, an infrared sensor, an ultrasonic sensor, and an RFID reader; the shelf layout information includes shelf position, shelf size, number of shelf layers, and distribution information of free cargo spaces in each shelf layer; the obstacle distribution information includes temporary obstacles in the warehouse. and the position and size of permanent obstacles, the temporary obstacles include logistics forklifts and temporarily stacked goods; the target image set is input into a preset object classification model to determine the target classification corresponding to the target object; the object classification model adopts a deep learning model; at least one candidate idle cargo space matching the target classification is determined, and the target transportation path of the target object is determined according to each of the candidate idle cargo space, the real-time warehouse environment data and the target classification; a target gripping strategy matching the target classification is determined; the target path and the target gripping strategy are respectively sent to the AGV and the robotic arm, so that the AGV and the robotic arm cooperate to sort the target object.

[0006] Optionally, determining the target clamping strategy that matches the target classification includes: determining a target clamper that matches the target classification from multiple clampers; determining the clamping working parameters of the clamper according to the target classification to obtain a corresponding target clamping strategy; the clamping working parameters include clamping force and clamper movement speed.

[0007] Optionally, determining the target transportation path and target clamping strategy of the target object according to each of the candidate free cargo spaces, the real-time warehouse environment data and the target classification includes: determining the target object transportation conditions that match the target classification from a set of object transportation conditions; the set of object transportation conditions includes multiple calibrated object classifications and corresponding transportation conditions, and the transportation conditions at least define a path obstacle density range and a path slope interval that match the calibrated object classification; for each of the candidate free cargo spaces, assuming the candidate free cargo space as the end point of the path, and processing the real-time warehouse environment data and the target object transportation conditions according to a dynamic path planning algorithm to plan a candidate transportation path with the minimum corresponding path total cost; comparing the total path costs corresponding to each of the candidate transportation paths, and determining the target free cargo space and the target transportation path according to the comparison result.

[0008] Optionally, the dynamic path planning algorithm adopts a conditional adaptive A* algorithm, and a unit cost function of the conditional adaptive A* algorithm is defined according to the path distance and the transportation condition of the target object;

[0009] The expression of the unit cost function C is:

[0010] C=α·D+β·M1+γ·M2

[0011] Where D represents the unit moving cost, M1 and M2 represent the matching degree of the planned path unit for the path obstacle density range and path slope range respectively, and α, β, and γ represent the weight parameters of different influencing factors.

[0012] Optionally, the conditional adaptive A* algorithm is:

[0013] f(n)=g(n)+h(n)

[0014] h(n)=ω·h ML (n)+(1-ω)·h static (n)

[0015]

[0016] Where f(n) represents the total cost of the path, g(n) represents the hypothetical actual cost of the target object moving from the starting location node to location node n, and h(n) represents the estimated cost of moving from location node n to the end point of the path corresponding to the candidate free cargo space; h ML (n) represents the heuristic cost predicted by the machine learning model; h static (n) represents the static heuristic cost, ω represents the heuristic cost weight factor; (x n ,y n ) and (x goal ,y goal ) represent the coordinates of the location node n and the end point of the path respectively.

[0017] Optionally, the object classification model comprises a deep residual network module, an SE network module and a classification network module; the deep residual network module is used to extract image features in the target image set; the deep residual network module comprises a plurality of residual blocks, each residual block comprises several convolutional layers;

[0018] The structure of the residual block is:

[0019] F(x,{W i})+x

[0020] In the formula, F(x,{W i}) represents the residual mapping achieved by the convolutional layer, W i represents the weight of the convolutional layer, and x represents the input feature of the residual block, wherein the input feature corresponding to the first residual block is determined according to the target image set;

[0021] The SE network module includes a plurality of SE blocks, each of which is inserted after each residual block in the deep residual network module to perform channel-level weight adjustment on a feature map output by each residual block;

[0022] The structure of the SE block is:

[0023]

[0024] Where F(x) c,h,w represents the activation value of the element (h,w) at a specific position in the feature map on the cth channel, h and w represent the position index on the height and width of the feature map respectively; H and W are the height and width of the feature map; S represents the average activation value of all spatial positions of the feature map on channel c through the squeezing operation; σ represents the sigmoid activation function, δ represents the ReLU activation function, and W1 and W2 are the weights of the fully connected layer; represents the output of the SE block at the cth channel;

[0025] The classification network module is connected to the last SE block and is used to perform classification prediction based on the features adjusted by the SE block weights to output the corresponding target classification;

[0026] The structure of the classification network module is:

[0027]

[0028] Where P(y=j|x) represents the probability of determining that an object belongs to category j based on the input target image set x, and z j It represents the output score of the jth category calculated by the classification layer, and K represents the total number of categories for object classification.

[0029] Optionally, the loss function of the object classification model is:

[0030]

[0031] In the formula, y k is the true label corresponding to the training sample. If the training sample belongs to category k, then y k =1, otherwise y k =0; p k is the probability of category k predicted by the model, w k represents the weight for category k, γ represents the preset adjustment focus parameter, m(k) represents the relative frequency of category k in the data set, and ζ represents a preset constant.

[0032] The present application also provides an efficient article sorting system for an intelligent warehousing system, comprising: an acquisition unit, for acquiring real-time warehousing environment data and a target image set for a target object to be sorted; the real-time warehousing environment data is environmental data in the warehouse collected in real time through a sensor network, and the real-time warehousing environment data includes shelf layout information and obstacle distribution information; the target image set includes images of the target object captured from multiple angles; the sensor network includes a laser scanner, an infrared sensor, an ultrasonic sensor, and an RFID reader; the shelf layout information includes shelf position, shelf size, number of shelf layers, and distribution information of free cargo spaces in each shelf layer; the obstacle distribution information includes the position and size of temporary obstacles and permanent obstacles in the warehouse. The temporary obstacles include logistics forklifts and temporarily stacked goods; a classification unit is used to input the target image set into a preset object classification model to determine the target classification corresponding to the target object; the object classification model adopts a deep learning model; a path determination unit is used to determine at least one candidate idle cargo location that matches the target classification, and determine the target transportation path of the target object according to each of the candidate idle cargo locations, the real-time warehouse environment data and the target classification; a strategy determination unit is used to determine a target clamping strategy that matches the target classification; a sending unit is used to send the target path and the target clamping strategy to the AGV and the robotic arm respectively, so that the AGV and the robotic arm cooperate to sort the target object.

[0033] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an efficient item sorting method for the intelligent warehousing system as described in any one of the above-mentioned methods is implemented.

[0034] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an efficient item sorting method for an intelligent warehousing system as described in any one of the above.

[0035] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements an efficient item sorting method for an intelligent warehousing system as described in any one of the above.

[0036] The efficient item sorting method and system for an intelligent warehousing system provided by the present application can produce at least the following technical effects:

[0037] (1) By using a deep learning model to classify objects in a target image set, the system can effectively identify various types of goods, including unmarked, irregularly shaped, or newly launched goods, achieving a more flexible and accurate sorting process than barcode recognition technology, supporting the sorting of uncoded goods, and being able to handle a wider variety of goods, suitable for a wider range of application scenarios, and can reduce the probability of manual sorting due to recognition errors, thereby reducing the cost of manual intervention. At the same time, compared with manual operations, the automated system has a lower error rate, which improves the accuracy and reliability of the operation.

[0038] (2) By using real-time warehouse environment data, including shelf layout and obstacle distribution information, the system can intelligently plan transportation routes and free cargo spaces for the target objects to be sorted. The system can determine the optimal transportation route and gripping strategy for the automated guided vehicle (AGV) and robotic arm. This reduces logistics congestion and detours caused by unreasonable shelf layout or temporary obstacles, and also reduces the navigation difficulty of AGVs in complex environments. It achieves a comprehensive and holistic consideration of the various transfer processes of the target objects from the warehouse starting point to the shelf, thereby improving the overall sorting efficiency and accuracy.

[0039] (3) After the target classification of the items to be sorted is automatically identified through the artificial intelligence model, the gripping strategy is determined based on the target classification to ensure a high degree of match between the gripping strategy and the sorting object, such as whether to use a gripper that matches the appearance of the item to ensure the reliability of object sorting. In addition, when planning the target transportation path, it is necessary to comprehensively consider the target classification to ensure that it can be transported to the candidate vacant cargo space that matches the corresponding item classification, so that the target transportation path can meet the transportation requirements of the target object of the corresponding target classification, meet the personalized transportation and sorting requirements for various types of items, and realize the orderly sorting of various types of objects to matching shelf locations without the need for manual secondary sorting, thereby reducing the manual sorting cost of shelves.

[0040] Through this technical solution, in the item sorting system, by integrating sensor networks and deep learning models, guiding the collaborative work of AGV and robotic arms, the intelligence level of the intelligent warehousing system is significantly improved, and complex sorting tasks can be completed more intelligently and automatically. As a result, not only the sorting accuracy and efficiency are improved, labor costs and human errors are reduced, but also the system's ability to handle different items and its intelligence level are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart showing an example of an efficient item sorting method for an intelligent warehousing system according to an embodiment of the present application;

[0043] Figure 2 Shown according to Figure 1 An exemplary operation flow chart of step S130 in FIG.

[0044] Figure 3 A schematic diagram showing the structure of an example of an object classification model according to an embodiment of the present application is shown;

[0045] Figure 4 A structural block diagram of an example of an efficient item sorting system of an intelligent warehousing system according to an embodiment of the present application is shown;

[0046] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] Figure 1 A flowchart showing an example of an efficient item sorting method for an intelligent warehousing system according to an embodiment of the present application is shown.

[0049] Regarding the execution subject of the method of the embodiment of the present application, it can be any electronic device with processing and computing capabilities, such as a computer terminal, a mobile phone terminal, etc. More specifically, in combination with the business scenario of the item sorting management platform, its execution subject can be a central server to achieve full-process automated sorting of target objects from the starting point of the warehouse to the shelf by integrating sensor networks and deep learning models to guide the collaborative work of AGVs and robotic arms, thereby improving sorting accuracy and efficiency.

[0050] like Figure 1 As shown, in step S110, real-time warehouse environment data and a target image set for the target object to be sorted are obtained.

[0051] Here, the real-time warehouse environment data is the environment data in the warehouse collected in real time through the sensor network. The real-time warehouse environment data includes shelf layout information and obstacle distribution information, and the target image set includes images of target objects captured from multiple angles. Therefore, the real-time collection of the real-time warehouse environment data and obstacle distribution information is achieved through the sensor network and high-definition camera device.

[0052] In some examples of the embodiments of the present application, the sensor network includes laser scanners, infrared sensors, ultrasonic sensors, and RFID readers. The shelf layout information includes shelf location, shelf size, number of shelf layers, and distribution information of free cargo spaces in each shelf layer. The obstacle distribution information includes the location and size of temporary obstacles and permanent obstacles in the warehouse. Temporary obstacles include logistics forklifts and temporarily stacked goods.

[0053] More specifically, by using a laser scanner to measure the position and size of shelves and obstacles, using an infrared sensor to detect the existence of obstacles and their approximate positions, and using an ultrasonic sensor to assist in detecting the distance between obstacles and objects, it is possible to supplement and improve the position measurement information when the line of sight is obstructed, ensuring the high accuracy of the position information and distance information of the measured objects. In addition, RFID tags are also set on the shelves, and each RFID tag is scanned by an RFID reader to obtain the real-time cargo stacking information of each shelf, so as to obtain the distribution information of the free cargo spaces on each shelf. Furthermore, the item sorting management platform can complete the scheduling and allocation of each object to be sorted and the corresponding free cargo spaces according to the distribution of the free cargo spaces.

[0054] In step S120, the target image set is input into a preset object classification model to determine the target classification corresponding to the target object, and the object classification model adopts a deep learning model.

[0055] Here, the object classification model can adopt various types of deep learning models, such as convolutional neural networks, transfer learning networks, etc., which are not limited here. More preferably, the deep learning model can adopt other more novel architectures to process the target image set and realize the classification of objects. More details will be expanded in conjunction with other examples below.

[0056] Here, by using a deep learning model to classify objects in a target image set, the system can effectively identify various types of goods, including unmarked, irregularly shaped, or newly launched goods, achieving a more flexible and accurate sorting process than barcode recognition technology, supporting the sorting of uncoded goods, and being able to handle a wider variety of goods, suitable for a wider range of application scenarios, and can reduce the probability of manual sorting due to recognition errors, reducing the cost of manual intervention. At the same time, the automated system has a lower error rate than manual operations, improving the accuracy and reliability of operations. In addition, deep learning models have the ability to learn from large amounts of data, so they can better handle noise in images and slight changes in items, improving the overall robustness of the system.

[0057] In step S130, at least one candidate free cargo space matching the target classification is determined, and a target transportation path of the target object is determined based on each candidate free cargo space, the real-time warehouse environment data, and the target classification.

[0058] In some examples of the embodiments of the present application, a plurality of shelves are provided in the warehouse, and different shelves (or different layers of shelves) are used to stack corresponding types of items. Here, after obtaining the target classification of the objects to be sorted through the deep learning model, the item sorting management platform can query (for example, look up a table) to obtain at least one candidate free cargo space that matches the target classification. Then, path planning is performed according to the path planning algorithm to determine the corresponding target transportation path. As a result, the system can determine the optimal transportation path and clamping strategy for the AGV and the robotic arm, reduce the navigation difficulty of the AGV in a complex environment, realize the comprehensive overall consideration of the various transfer processes of the target object from the warehouse starting point to the shelf, and improve the overall sorting efficiency and accuracy.

[0059] According to the first example of the embodiment of the present application, a target cargo location for accommodating the target object is selected arbitrarily or conditionally from each candidate free cargo location, for example, the free cargo location closest to the warehouse exit is preferentially selected to obtain the target cargo location, which is used as the end point, and path planning is performed to obtain the corresponding target transportation path. According to the second example of the embodiment of the present application, the corresponding transportation paths are planned according to each candidate free cargo location, and the matching degree between each transportation path and the target classification is compared. For example, if the object to be sorted is a fragile item, it should be more suitable for walking on a flat and wide road and should avoid choosing a crowded section or bypassing a steep slope, so as to obtain the final target transportation path to meet the personalized transportation needs of the items.

[0060] In step S140, a target clamping strategy matching the target classification is determined.

[0061] Specifically, after the target classification of the items to be sorted is automatically identified through the artificial intelligence model, the clamping strategy is determined based on the target classification to ensure a high degree of match between the clamping strategy and the sorting object, such as whether to use a clamp that matches the appearance of the item to ensure the reliability of object sorting. In addition, when planning the target transportation path, it is necessary to comprehensively consider the target classification to ensure that it can be transported to the candidate vacant cargo space that matches the corresponding item classification, so that the target transportation path can meet the transportation requirements of the target object of the corresponding target classification, meet the personalized transportation and sorting needs for various types of items, and realize the orderly sorting of various types of objects to matching shelf positions without manual secondary sorting, thereby reducing the cost of manual sorting of shelves.

[0062] In some examples of the embodiments of the present application, a target gripper that matches the target classification is determined from a plurality of grippers. Further, the gripping working parameters of the gripper are determined according to the target classification to obtain a corresponding target gripping strategy, and the gripping working parameters include gripping force and gripper moving speed. For example, considering the stability and safety of object sorting, for fragile objects, flexible grippers or grippers with cushioning materials can be used. In addition, the gripping force and gripper moving speed should be appropriate to avoid the object from breaking during the gripping and moving process; in addition, for irregularly shaped or large objects, special gripping mechanisms or multi-point gripping may be required to meet the gripping and transportation requirements of various objects.

[0063] In step S150, the target path and the target gripping strategy are sent to the AGV and the robotic arm respectively, so that the AGV and the robotic arm cooperate to sort the target objects.

[0064] In some embodiments, the item sorting management platform sends the target path to the AGV, so that the AGV transports the target object from the starting position to the end position, which can be an agreed position close to the target shelf. The item sorting management platform then sends the target clamping strategy to the robotic arm, so that the robotic arm can clamp the target object and place it in the target cargo location. In this way, the target object can be automatically transported and sorted from the starting point to the cargo location through the collaboration of the AGV and the robotic arm.

[0065] Through the embodiments of the present application, in the item sorting system, by integrating the sensor network and the deep learning model, the coordinated work of the AGV and the robotic arm is guided, which significantly improves the intelligence level of the intelligent warehousing system, and can complete complex sorting tasks more intelligently and automatically, thereby enhancing the intelligence level of the system. As a result, not only the sorting accuracy and efficiency are improved, the labor cost and human errors are reduced, but also the system's ability to handle different items and its intelligence level are enhanced.

[0066] Figure 2 Shown according to Figure 1An example operation flow chart of step S130 in FIG.

[0067] like Figure 2 As shown, in step S210, target object transportation conditions matching the target classification are determined from the object transportation condition set.

[0068] Here, the object transportation condition set includes multiple calibration object classifications and corresponding transportation conditions, and the transportation conditions at least define the path obstacle density range and path slope interval that match the calibration object classification. Exemplarily, the path obstacle density range and path slope interval corresponding to the calibration object classification "glass products" are A1 and B1, respectively, and the path obstacle density range and path slope interval corresponding to the calibration object classification "plastic products" are A2 and B2, respectively, where A1<A2, B1<B2, thereby realizing personalized transportation solutions for different types of objects.

[0069] In step S220, for each candidate free cargo location, the candidate free cargo location is assumed to be the end point of the path, and the real-time warehouse environment data and the target object transportation conditions are processed according to the dynamic path planning algorithm to plan a candidate transportation path with the minimum total cost of the corresponding path.

[0070] It should be noted that, considering that the warehouse environment may change frequently, a dynamic path planning algorithm (e.g., A*, D*Lite or RRT*) that can respond to environmental changes in real time is selected to adjust the planned path in real time according to the node position of the object movement, so as to ensure the reliability of the object transportation. It should also be pointed out that in this embodiment, each candidate free cargo space suitable for placing the target object is assumed to be the end point of the path, and the corresponding path planning calculation with the minimum cost is performed. In other words, the transportation end point of the target object (i.e., the target free cargo space) may be floating during the transportation process, and it changes with the minimum cost calculation result, so as to obtain a more dynamic and diversified transportation sorting scheme, and realize transportation sorting with the minimum comprehensive cost.

[0071] In addition, by taking the target object transportation conditions and real-time warehouse environment data that match the objects to be sorted as the input parameters of the dynamic path planning algorithm, it is possible to achieve the path planning while avoiding obstacles and efficiently reaching the target cargo location, and also meet the personalized transportation and sorting needs for target classification.

[0072] In step S230, the total path costs corresponding to the candidate transportation paths are compared, and the target free cargo space and the target transportation path are determined according to the comparison result.

[0073] In some embodiments, each candidate idle cargo location is assumed to be the end point, and the corresponding minimum path total cost is calculated according to the dynamic path planning algorithm. After obtaining the minimum path total cost corresponding to each candidate idle cargo location, the minimum path total costs are compared, and the candidate idle cargo location and candidate transportation path corresponding to the minimum one are determined as the target idle cargo location and target transportation path selected by the current node. Specifically, every time the AGV moves through a preset number of cell nodes, it refreshes and iterates the calculation of the corresponding target idle cargo location and target transportation path to achieve dynamic path planning according to the real-time warehouse environment and ensure that the path cost is minimized.

[0074] Furthermore, considering the diversity of object types and the differences in the transportation conditions required by different types of objects during transportation, for example, "glass products" need to be transported more stably, while "plastic products" need to be transported more efficiently. Here, the dynamic path planning algorithm adopts the conditional adaptive A* algorithm (AC-A* algorithm), and the unit cost function of the conditional adaptive A* algorithm is defined according to the path distance and the transportation conditions of the target object.

[0075] In some embodiments, the unit cost function may be determined by weighting various influencing factors. For example, the expression of the unit cost function C is as follows:

[0076] C=α·D+β·M1+γ·M2 Formula (1)

[0077] Where D represents the unit moving cost, M1 and M2 represent the matching degree of the planned path unit for the path obstacle density range and path slope range respectively, and α, β, and γ represent the weight parameters of different influencing factors.

[0078] Therefore, when using the conditional adaptive A* algorithm to calculate the path cost, in addition to the path distance, the matching degree between the planned path and the transportation conditions of the target object can also have an impact to ensure that the objects can be transported and sorted quickly and safely.

[0079] In some examples of the embodiments of the present application, the conditional adaptive A* algorithm is:

[0080] f(n)=g(n)+h(n) Formula (2)

[0081] h(n)=ω·h ML (n)+(1-ω)·h static (n) Formula (3)

[0082]

[0083] In formula (2), the total path cost function f(n) balances the evaluation of the path taken and the prediction of the future path by combining g(n) and h(n), which can effectively guide the search process towards the target while considering the total cost of reaching the target. The heuristic function h(n) provides information about the future path, helping the algorithm make better choices among possible paths, which can significantly improve the search efficiency.

[0084] In formula (3), a machine learning model is used to predict the heuristic cost h ML (n), and then updating h(n), which can effectively change the search behavior of the algorithm and make it more focused on the path with the lowest cost.

[0085] In formula (4), the Euclidean distance is used as the static heuristic cost. Compared with the Manhattan distance, it can support the displacement of AGV in more directions, making its movement path more flexible and not limited to the grid axial displacement.

[0086] Where f(n) represents the total cost of the path, g(n) represents the hypothetical actual cost of the target object moving from the starting location node to location node n, and h(n) represents the estimated cost of moving from location node n to the end point of the path corresponding to the candidate free cargo space; h ML (n) represents the heuristic cost predicted by the machine learning model; h static (n) represents the static heuristic cost, ω represents the heuristic cost weight factor; (x n ,y n ) and (x goal ,y goal ) represent the coordinates of the location node n and the end point of the path respectively.

[0087] It should be noted that the conditional adaptive A* algorithm according to the above formulas (2)-(4) is an enhancement of the classic A* algorithm. It dynamically adjusts the heuristic function by introducing a machine learning model (for example, DNN, etc.) to better adapt to the dynamically changing environment. In particular, by learning the laws of warehouse environment changes, the AC-A* algorithm can quickly adjust the path, achieve efficient and accurate path planning, and improve the efficiency and accuracy of logistics operations.

[0088] Here, the machine learning model can adopt various general or potential machine learning models, and can use historical path data to train the machine learning model so that it can predict the estimated cost h from any node n to the target. ML(n). More specifically, the input data or feature dimensions of the machine learning model can be determined based on the real-time warehouse environment data to express the location of the current node, the target location, the layout of the surrounding environment, the historical congestion of the area, etc. In this way, as the warehouse environment continues to change and more data is accumulated, the machine learning model is regularly updated to maintain its accuracy and adaptability. The sample label information of the machine learning model can be determined based on the actual cost of each path to intuitively reflect the transportation efficiency of the sorting path and the matching degree between the sorting path and the target object.

[0089] Figure 3 A schematic structural diagram of an example of an object classification model according to an embodiment of the present application is shown.

[0090] like Figure 3 As shown, the input of the object classification model 300 is a target image set, i.e., a preprocessed warehouse item image, and its output is a target classification, i.e., a probability distribution of the category to which the item belongs. The object classification model 300 includes a deep residual network module 310, a SE network module 320, and a classification network module 330.

[0091] More specifically, the deep residual network module 310 is used to extract image features in the target image set, and the deep residual network module includes multiple residual blocks, each of which includes several convolutional layers.

[0092] The structure of the residual block is:

[0093] F(x,{W i})+x formula (5)

[0094] In the formula, F(x,{W i}) represents the residual mapping achieved by the convolutional layer, W i Represents the weight of the convolutional layer, x represents the input feature of the residual block, where the input feature corresponding to the first residual block is determined according to the target image set.

[0095] The SE network module 320 includes multiple SE blocks, each of which is inserted after each residual block in the deep residual network module to perform channel-level weight adjustment on the feature map output by each residual block.

[0096] The structure of the SE block is:

[0097]

[0098] Where F(x) c,h,wrepresents the activation value of the element (h,w) at a specific position in the feature map on the cth channel, h and w represent the position index on the height and width of the feature map respectively; H and W are the height and width of the feature map; S represents the average activation value of all spatial positions of the feature map on channel c through the squeezing operation; σ represents the sigmoid activation function, δ represents the ReLU activation function, and W1 and W2 are the weights of the fully connected layer; represents the output of the SE block at the cth channel.

[0099] The classification network module 330 is connected to the last SE block and is used to perform classification prediction based on the features adjusted by the SE block weights to output the corresponding target classification.

[0100] The structure of the classification network module 330 is:

[0101]

[0102] Where P(y=j|x) represents the probability of determining that an object belongs to category j based on the input target image set x, and z j It represents the output score of the jth category calculated by the classification layer, and K represents the total number of categories for object classification.

[0103] Based on the above processing flow, features are extracted through the deep residual network module 310, and the importance of the features at the channel level is adjusted using the SE network module 320. Finally, the probability of each category is output through the classification network module 330 to accurately identify and classify various items in the warehouse.

[0104] It should be noted that in a warehouse environment, the shapes, sizes, and appearances of items may vary greatly. The model designed with this structure can effectively cope with the diversity of classification targets while maintaining efficient and accurate classification performance. Specifically, after extracting complex image features using a deep network structure, the introduced SE block emphasizes important feature channels through an attention mechanism, allowing the model to focus on information that is most helpful for classification, thereby improving classification accuracy. In addition, the attention mechanism reduces the processing of irrelevant information by focusing on key features, thereby improving the processing efficiency of the model.

[0105] Furthermore, in combination with the object classification scenario of the intelligent warehousing system of the embodiment of the present application, in order to improve the classification performance and training efficiency of the model, especially when facing the classification processing of multi-category and unbalanced data sets, the embodiment of the present application provides a loss function that combines cross entropy loss and focal loss, namely, the weighted focal cross entropy loss function.

[0106] In some examples of the embodiments of the present application, the loss function of the object classification model is:

[0107]

[0108] In the formula, y k is the true label corresponding to the training sample. If the training sample belongs to category k, then y k =1, otherwise y k =0; p k is the probability of category k predicted by the model, w k represents the weight for category k, γ represents the preset adjustment focus parameter, m(k) represents the relative frequency of category k in the data set, and ζ represents a preset constant. Here, ζ is a small constant (e.g., 1.02) used to ensure numerical stability.

[0109] In the loss function provided in the embodiment of the present application, in order to solve the problem of class imbalance, each class k is assigned a weight w k , which is inversely proportional to the frequency of occurrence of the class. The focus parameter γ is used to reduce the relative loss of easily classified samples and increase the penalty for difficult or misclassified samples.

[0110] Here, γ is mainly used to reduce the relative contribution of easy-to-classify samples while increasing attention to hard-to-classify samples. The determination process for γ can be diverse. For example, the typical initial value of γ can start from 2.0 based on experience. If it is found that the model performs poorly for hard-to-classify samples (for example, the accuracy of some minority categories is low), you can try to increase the value of γ. On the contrary, if the model over-punishes easy-to-classify samples, resulting in a decrease in overall performance, the value of γ can be reduced.

[0111] Through the embodiments of the present application, in a warehousing system, some items may appear more frequently than other items. The weighted focal cross entropy loss is used to assign different weights to samples of different categories to solve this imbalance and ensure that the model does not favor more common categories. Through the focus parameters, the model pays more attention to samples that are difficult to classify (i.e., samples with a low model prediction probability), which helps to improve the model's ability to recognize complex or uncommon items. As a result, the object classification model can more effectively handle category imbalance in the intelligent warehousing system, focusing on improving the recognition ability of difficult-to-classify samples, thereby improving the overall classification performance and system operation efficiency.

[0112] The efficient item sorting system of the intelligent warehousing system provided by the present application is described below. The efficient item sorting system of the intelligent warehousing system described below and the efficient item sorting method of the intelligent warehousing system described above can be referred to each other.

[0113] Figure 4 A structural block diagram of an example of an efficient item sorting system for an intelligent warehousing system according to an embodiment of the present application is shown.

[0114] like Figure 4 As shown, an efficient article sorting system 400 of an intelligent warehousing system includes an acquisition unit 410, a classification unit 420, a path determination unit 430, a strategy determination unit 440 and a sending unit 450.

[0115] The acquisition unit 410 is used to acquire real-time warehouse environment data and a target image set for the target object to be sorted; the real-time warehouse environment data is the environment data in the warehouse collected in real time through a sensor network, and the real-time warehouse environment data includes shelf layout information and obstacle distribution information; the target image set includes images of the target object captured from multiple angles; the sensor network includes a laser scanner, an infrared sensor, an ultrasonic sensor and an RFID reader; the shelf layout information includes the shelf position, shelf size, number of shelf layers, and distribution information of vacant cargo spaces on each shelf layer; the obstacle distribution information includes the position and size of temporary obstacles and permanent obstacles in the warehouse, and the temporary obstacles include logistics forklifts and temporarily stacked goods.

[0116] The classification unit 420 is used to input the target image set into a preset object classification model to determine the target classification corresponding to the target object; the object classification model adopts a deep learning model.

[0117] The path determination unit 430 is used to determine at least one candidate idle cargo location that matches the target classification, and determine the target transportation path of the target object according to each of the candidate idle cargo locations, the real-time warehouse environment data and the target classification.

[0118] The strategy determination unit 440 is used to determine a target clamping strategy that matches the target classification.

[0119] The sending unit 450 is used to send the target path and the target clamping strategy to the AGV and the robotic arm respectively, so that the AGV and the robotic arm cooperate to sort the target object.

[0120] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by electronic devices (including but not limited to computers, servers, or network devices, etc.) to execute the efficient item sorting method of the above-mentioned intelligent warehousing system of the present application.

[0121] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned efficient item sorting method for the intelligent warehousing system.

[0122] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an efficient item sorting method for an intelligent warehousing system.

[0123] Figure 5 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for executing an efficient item sorting method for an intelligent warehousing system provided by another embodiment of the present application. Figure 5 As shown, the device includes:

[0124] One or more processors 510 and memory 520, Figure 5 A processor 510 is taken as an example.

[0125] The device for executing the efficient item sorting method of the intelligent warehousing system may further include: an input device 530 and an output device 540 .

[0126] The processor 510, the memory 520, the input device 530 and the output device 540 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0127] The memory 520 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the efficient item sorting method for the intelligent warehousing system in the embodiment of the present application. The processor 510 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 520, that is, realizing the efficient item sorting method for the intelligent warehousing system in the above method embodiment.

[0128] The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may optionally include a memory remotely arranged relative to the processor 510, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The input device 530 may receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 540 may include a display device such as a display screen.

[0130] The one or more modules are stored in the memory 520, and when executed by the one or more processors 510, perform the efficient item sorting method for the intelligent warehousing system in any of the above method embodiments.

[0131] The above-mentioned product can execute the efficient item sorting method of the intelligent warehousing system provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present application.

[0132] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0133] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0134] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0135] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0136] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An efficient item sorting method for an intelligent warehousing system, comprising: Acquire real-time warehouse environment data and target image sets for target objects to be sorted; The real-time warehouse environment data is the environment data in the warehouse collected in real time through the sensor network, and the real-time warehouse environment data includes shelf layout information and obstacle distribution information; the target image set includes images of the target object captured from multiple angles; the sensor network includes laser scanners, infrared sensors, ultrasonic sensors and RFID readers; the shelf layout information includes shelf location, shelf size, number of shelf layers, and distribution information of free cargo spaces in each shelf layer; the obstacle distribution information includes the location and size of temporary obstacles and permanent obstacles in the warehouse, and the temporary obstacles include logistics forklifts and temporarily stacked goods; Inputting the target image set into a preset object classification model to determine the target classification corresponding to the target object; The object classification model adopts a deep learning model; Determine at least one candidate free cargo space that matches the target classification, and determine a target transportation path for the target object based on each of the candidate free cargo spaces, the real-time warehouse environment data, and the target classification; determining a target gripping strategy that matches the target classification; The target path and the target clamping strategy are respectively sent to the AGV and the robotic arm, so that the AGV and the robotic arm cooperate to sort the target object.

2. The method according to claim 1, wherein: The determining of a target clamping strategy that matches the target classification comprises: Determining, from a plurality of grippers, a target gripper that matches the target classification; The clamping working parameters of the clamper are determined according to the target classification to obtain a corresponding target clamping strategy; the clamping working parameters include clamping force and clamper moving speed.

3. The method according to claim 1, wherein: The step of determining the target transportation path and the target clamping strategy of the target object according to each of the candidate free cargo spaces, the real-time warehouse environment data and the target classification includes: Determining a target object transportation condition matching the target classification from an object transportation condition set; the object transportation condition set includes a plurality of calibration object classifications and corresponding transportation conditions, wherein the transportation conditions at least define a path obstacle density range and a path slope interval matching the calibration object classification; For each candidate free cargo space, the candidate free cargo space is assumed to be a path end point, and the real-time warehouse environment data and the target object transportation conditions are processed according to a dynamic path planning algorithm to plan a candidate transportation path with the minimum total cost of the corresponding path; The total path costs corresponding to the candidate transportation paths are compared, and the target free cargo space and the target transportation path are determined according to the comparison result.

4. The method according to claim 3, wherein: The dynamic path planning algorithm adopts a conditional adaptive A* algorithm, and the unit cost function of the conditional adaptive A* algorithm is defined according to the path distance and the transportation condition of the target object; The expression of the unit cost function C is: C=α·D+β·M1+γ·M2 Where D represents the unit moving cost, M1 and M2 represent the matching degree of the planned path unit for the path obstacle density range and path slope range respectively, and α, β, and γ represent the weight parameters of different influencing factors.

5. The method according to claim 4, wherein: The conditional adaptive A* algorithm is: f(n)=g(n)+h(n) h(n)=ω·h ML (n)+(1-ω)·h static (n) Where f(n) represents the total cost of the path, g(n) represents the hypothetical actual cost of the target object moving from the starting location node to location node n, and h(n) represents the estimated cost of moving from location node n to the end point of the path corresponding to the candidate free cargo space; h ML (n) represents the heuristic cost predicted by the machine learning model; h static (n) represents the static heuristic cost, ω represents the heuristic cost weight factor; (x n ,y n ) and (x goal ,y goal ) represent the coordinates of the location node n and the end point of the path respectively.

6. The method according to any one of claims 1 to 5, wherein: The object classification model includes a deep residual network module, an SE network module and a classification network module; The deep residual network module is used to extract image features in the target image set; the deep residual network module includes a plurality of residual blocks, each residual block includes several convolutional layers; The structure of the residual block is: F(x,{W i })+x In the formula, F(x,{W i }) represents the residual mapping achieved by the convolutional layer, W i represents the weight of the convolutional layer, and x represents the input feature of the residual block, wherein the input feature corresponding to the first residual block is determined according to the target image set; The SE network module includes a plurality of SE blocks, each of which is inserted after each residual block in the deep residual network module to perform channel-level weight adjustment on a feature map output by each residual block; The structure of the SE block is: In the formula, F(x) c,h,w represents the activation value of the element (h,w) at a specific position in the feature map on the cth channel, where h and w represent the position index in the height and width of the feature map respectively; H and W are the height and width of the feature map; S represents the average activation value of all spatial positions of the feature map on channel c through the squeezing operation; σ represents the sigmoid activation function, δ represents the ReLU activation function, and W1 and W2 are the weights of the fully connected layer; represents the output of the SE block at the cth channel; The classification network module is connected to the last SE block and is used to perform classification prediction based on the features adjusted by the SE block weights to output the corresponding target classification; The structure of the classification network module is: Where P(y=j|x) represents the probability of determining that an object belongs to category j based on the input target image set x, and z j It represents the output score of the jth category calculated by the classification layer, and K represents the total number of categories for object classification.

7. The method according to claim 6, wherein: The loss function of the object classification model is: In the formula, y k is the true label corresponding to the training sample. If the training sample belongs to category k, then y k =1, otherwise y k =0; p k is the probability of category k predicted by the model, w k represents the weight for category k, γ represents the preset adjustment focus parameter, m(k) represents the relative frequency of category k in the data set, and ζ represents a preset constant.

8. An efficient item sorting system for an intelligent warehousing system, comprising: An acquisition unit, used to acquire real-time warehouse environment data and a target image set for a target object to be sorted; The real-time warehouse environment data is the environment data in the warehouse collected in real time through the sensor network, and the real-time warehouse environment data includes shelf layout information and obstacle distribution information; the target image set includes images of the target object captured from multiple angles; the sensor network includes laser scanners, infrared sensors, ultrasonic sensors and RFID readers; the shelf layout information includes shelf location, shelf size, number of shelf layers, and distribution information of free cargo spaces in each shelf layer; the obstacle distribution information includes the location and size of temporary obstacles and permanent obstacles in the warehouse, and the temporary obstacles include logistics forklifts and temporarily stacked goods; A classification unit, used for inputting the target image set into a preset object classification model to determine the target classification corresponding to the target object; The object classification model adopts a deep learning model; A path determination unit, configured to determine at least one candidate free cargo space that matches the target classification, and determine a target transportation path for the target object according to each of the candidate free cargo space, the real-time warehouse environment data, and the target classification; A strategy determination unit, used to determine a target clamping strategy that matches the target classification; The sending unit is used to send the target path and the target clamping strategy to the AGV and the robotic arm respectively, so that the AGV and the robotic arm cooperate to sort the target object.

Images