Return and supply sorting method, device and system
By using image recognition models and cameras in warehousing logistics, the problems of low efficiency and high labor cost are solved, and a more efficient and automated sorting process is achieved.
Patent Information
- Application Number
- CN202310626680.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
In the field of warehousing and logistics, the missing or damaged facebooks in reverse supply return operations lead to the inability to effectively sort the supply return process, resulting in the stagnation of the supply return process. The existing secondary sorting relies on manual operations, which is inefficient and cost-effective.
Automatic sorting is achieved by setting up a camera on the assembly line to collect and use the trained image recognition model to identify category information. The model is constructed through supervised training and datasets, combining representative value screening and camera position adjustment to improve recognition accuracy and efficiency.
The automation level of the reverse return and sorting process has been improved, sorting efficiency has been improved, labor costs have been reduced, and the model of relying on manual sorting has been broken, achieving a more efficient return and sorting process.
Smart Images

Figure CN119067555A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of warehousing logistics, and in particular to a return sorting method, device and system. Background Art
[0002] In the actual reverse return operation, there is often a situation where the returned goods have no delivery note or the delivery note is damaged. Since the item information cannot be obtained from the items missing from the delivery note, it is impossible to sort the returned goods through the delivery note. Such items will be temporarily stored in the reverse warehouse as problematic items, and their return process will be suspended. Such items will be further processed later. One way of processing is that the reverse warehouse will try to trace the forward order information of the goods through information such as the storage time; another way is that the customer cannot receive the return payment and actively contact the customer service. At this time, the corresponding problematic goods are matched according to the customer's return initiation time, the category of the ordered goods and other information for traceability. No matter which of the above methods is used, it takes at least one to three days to process. During this period, the reverse warehouse needs to properly manage the problematic goods. Specifically, the reverse warehouse will sort the problematic goods again according to the category, so as to classify and store the problematic goods of different categories. Due to the particularity of reverse operation, the problematic goods cover many categories from fresh food to electronic products, and the time-consuming processing of problematic goods is uncertain, so it is very necessary to sort them again.
[0003] At present, the secondary sorting is carried out manually, which leads to low efficiency of return sorting and high labor cost investment. Summary of the invention
[0004] In view of this, the embodiments of the present invention provide a return sorting method, device and system to solve the technical problems of low return sorting efficiency and high manpower cost investment.
[0005] To achieve the above object, according to one aspect of an embodiment of the present invention, a return sorting method is provided, comprising:
[0006] Collect multiple images of returned items through cameras set at different positions on the production line;
[0007] Inputting the multiple images of the returned items into a trained image recognition model, thereby outputting the category information of the returned items; wherein the training process of the image recognition model includes: using the multiple images of the training items and the position information and category information of the training items marked in each image, performing supervised training on the image recognition model to update the model parameters, thereby obtaining a trained image recognition model;
[0008] The returned items are sorted for return according to the category information of the returned items.
[0009] Optionally, the training process of the image recognition model includes:
[0010] Constructing a data set, the data set including multiple images of training objects and location information and category information of the training objects marked in each image, and dividing the data set into a training set, a validation set, and a test set;
[0011] Performing supervised training on the image recognition model using the training set to update model parameters, thereby obtaining a trained image recognition model;
[0012] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera;
[0013] The trained image recognition model is tested using the test set.
[0014] Optionally, construct a dataset, including:
[0015] For each category, calculate the representative value of each item under the category, so as to screen out training items under the category;
[0016] Calculate the volume variance of each training item in each category, and sort the categories in descending order of volume variance;
[0017] Images corresponding to the various categories are obtained from an image database, thereby constructing a data set; wherein, the higher the category is in the ranking, the more images corresponding to the category are obtained from the image database.
[0018] Optionally, calculating the representative value of each item under the category, thereby screening out training items under the category, includes:
[0019] For each item, calculate the average daily outbound quantity and unit volume of the item in the past period of time;
[0020] Calculate the maximum average daily outbound volume, minimum average daily outbound volume, maximum single piece volume, and minimum single piece volume of the category in the past period of time;
[0021] For each item, calculate the representative value of the item based on the average daily shipment volume, unit volume of the item in the past period, the maximum average daily shipment volume, minimum average daily shipment volume, maximum unit volume, and minimum unit volume of the items in the category in the past period;
[0022] Items with representative values greater than or equal to a representative threshold are selected as training items under the category.
[0023] Optionally, the representative value of the item is calculated based on the average daily shipment quantity, unit volume of the item in the past period, the maximum average daily shipment quantity, the minimum average daily shipment quantity, the maximum unit volume, and the minimum unit volume of the items under the category in the past period, including:
[0024] The representative value of the item is calculated using the following formula:
[0025]
[0026] Among them, P i is the representative value of item i, O i is the average daily outbound volume of item i in the past period of time, V i is the volume of a single item i, O max , O min 、V max 、V min , They are the maximum daily average outbound quantity, minimum daily average outbound quantity, maximum single piece volume, minimum single piece volume and average single piece volume of the items in the category in the past period of time; a, b, c are custom parameters.
[0027] Optionally, the camera includes a first camera, a second camera and a third camera, the first camera is fixed directly above the assembly line, the second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail; the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line.
[0028] Optionally, using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera, including:
[0029] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, thereby obtaining the verification accuracy of the image recognition model;
[0030] The training objects corresponding to the other part of the data in the verification set are placed on the production line, and images of the training objects are respectively captured by the second camera and the third camera, and the images are input into the trained image recognition model, thereby outputting category information of the training objects, and comparing the category information of the objects with the other part of the data in the verification set, thereby obtaining the camera accuracy;
[0031] Divide the camera accuracy by the inspection accuracy to obtain an accuracy ratio;
[0032] The positions and / or angles of the second camera and the third camera are adjusted according to the accuracy ratio.
[0033] In addition, according to another aspect of an embodiment of the present invention, a return sorting device is provided, comprising:
[0034] A collection module, used to collect multiple images of returned items through cameras arranged at different positions on the assembly line;
[0035] A recognition module, used to input the multiple images of the returned items into a trained image recognition model, so as to output the category information of the returned items; wherein the training process of the image recognition model includes: using the multiple images of the training items and the position information and category information of the training items marked in each image, performing supervised training on the image recognition model to update the model parameters, so as to obtain a trained image recognition model;
[0036] The sorting module is used to sort the returned items according to the category information of the returned items.
[0037] Optionally, a training module is also included, for:
[0038] Constructing a data set, the data set including multiple images of training objects and location information and category information of the training objects marked in each image, and dividing the data set into a training set, a validation set, and a test set;
[0039] Performing supervised training on the image recognition model using the training set to update model parameters, thereby obtaining a trained image recognition model;
[0040] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera;
[0041] The trained image recognition model is tested using the test set.
[0042] Optionally, the training module is further used to:
[0043] For each category, calculate the representative value of each item under the category, so as to screen out training items under the category;
[0044] Calculate the volume variance of each training item in each category, and sort the categories in descending order of volume variance;
[0045] Images corresponding to the various categories are obtained from an image database, thereby constructing a data set; wherein, the higher the category is in the ranking, the more images corresponding to the category are obtained from the image database.
[0046] Optionally, the training module is further used to:
[0047] For each item, calculate the average daily outbound quantity and unit volume of the item in the past period of time;
[0048] Calculate the maximum average daily outbound volume, minimum average daily outbound volume, maximum single piece volume, and minimum single piece volume of the category in the past period of time;
[0049] For each item, calculate the representative value of the item based on the average daily shipment volume, unit volume of the item in the past period, the maximum average daily shipment volume, minimum average daily shipment volume, maximum unit volume, and minimum unit volume of the items in the category in the past period;
[0050] Items with representative values greater than or equal to a representative threshold are selected as training items under the category.
[0051] Optionally, the training module is further used to:
[0052] The representative value of the item is calculated using the following formula:
[0053]
[0054] Among them, P i is the representative value of item i, O i is the average daily outbound volume of item i in the past period of time, V i is the volume of a single item i, O max , O min 、V max 、V min , They are the maximum daily average outbound quantity, minimum daily average outbound quantity, maximum single piece volume, minimum single piece volume and average single piece volume of the items in the category in the past period of time; a, b, c are custom parameters.
[0055] Optionally, the camera includes a first camera, a second camera and a third camera, the first camera is fixed directly above the assembly line, the second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail; the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line.
[0056] Optionally, the training module is further used to:
[0057] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, thereby obtaining the verification accuracy of the image recognition model;
[0058] The training objects corresponding to the other part of the data in the verification set are placed on the production line, and images of the training objects are respectively captured by the second camera and the third camera, and the images are input into the trained image recognition model, thereby outputting category information of the training objects, and comparing the category information of the objects with the other part of the data in the verification set, thereby obtaining the camera accuracy;
[0059] Divide the camera accuracy by the inspection accuracy to obtain an accuracy ratio;
[0060] The positions and / or angles of the second camera and the third camera are adjusted according to the accuracy ratio.
[0061] In addition, according to another aspect of an embodiment of the present invention, a return sorting system is provided, comprising the return sorting device described in the above embodiment and cameras arranged at different positions on the assembly line.
[0062] Optionally, it further comprises a first slide rail and a second slide rail, wherein the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line;
[0063] The cameras include a first camera, a second camera and a third camera. The first camera is fixed directly above the assembly line. The second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail.
[0064] Optionally, it also includes:
[0065] The robot arm is used to automatically sort the returned items according to the category information of the returned items.
[0066] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:
[0067] one or more processors;
[0068] a storage device for storing one or more programs,
[0069] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.
[0070] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above embodiments is implemented.
[0071] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.
[0072] One embodiment of the above invention has the following advantages or beneficial effects: because a plurality of images of returned items are input into a trained image recognition model, thereby outputting the category information of the returned items, and sorting the returned items according to the category information of the returned items, the technical problems of low return sorting efficiency and large labor cost investment in the prior art are overcome. The embodiment of the present invention identifies the category information of the returned items through the image recognition model, improves the automation level of the reverse return sorting process, breaks the original operation mode of secondary sorting of problematic items by manual sorting, improves the efficiency of the return sorting process, and reduces the labor cost of reverse return sorting.
[0073] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0075] Figure 1 is a flow chart of a return sorting method according to an embodiment of the present invention;
[0076] Figure 2 is a schematic diagram of the installation positions of various cameras according to an embodiment of the present invention;
[0077] Figure 3 is a schematic diagram of the structure of an image recognition model according to an embodiment of the present invention;
[0078] Figure 4 is a flow chart of a return sorting method according to a reference embodiment of the present invention;
[0079] Figure 5 is a flow chart of a return sorting method according to another reference embodiment of the present invention;
[0080] Figure 6 is a schematic diagram of a return sorting device according to an embodiment of the present invention;
[0081] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;
[0082] Figure 8 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION
[0083] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0084] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.
[0085] Figure 1 is a flow chart of a return sorting method according to an embodiment of the present invention. As an embodiment of the present invention, Figure 1 As shown, the return sorting method may include:
[0086] Step 101, collecting multiple images of returned items by using cameras arranged at different positions on the production line.
[0087] In an embodiment of the present invention, scenarios in which the return supply sorting method is applied include but are not limited to reverse return supply scenarios. In reverse warehouse sorting, multiple cameras can be set at different positions on the assembly line, and multiple images of the returned items are collected by these cameras. If the shapes of the items processed by a return warehouse are relatively regular and uniform, in order to increase the recognition difficulty of the image recognition model, cameras are usually installed at multiple angles on the assembly line for image capture to obtain effective input images; conversely, if the shapes of the items processed by a return warehouse are relatively irregular, 1-2 cameras can be used for image capture. It should be noted that image capture at too many angles not only increases the cost of the equipment, but excessive input will also affect the speed of image recognition.
[0088] Optionally, the camera comprises a first camera, a second camera and a third camera, the first camera is fixed directly above the assembly line, the second camera and the third camera are slidably connected to a first slide rail and a second slide rail respectively, and can move along the first slide rail and the second slide rail; the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line. Figure 2 As shown, camera A, camera B and camera C are arranged on the assembly line, wherein camera A is located and fixed directly above the assembly line 10, so that camera A can capture images directly above the returned items, camera B and camera C are slidingly connected to the first slide rail 20 and the second slide rail 30 respectively, the first slide rail 20 and the second slide rail 30 are symmetrically located on both sides above the assembly line 10, and camera B can move along the first slide rail 20, and camera C can move along the second slide rail 30.
[0089] It should be noted that during the model training stage, the installation position and installation angle of camera B and camera C can be adjusted to accurately identify the category information of the returned items. It should be noted that the installation position of camera A is preferably close to the robotic arm. After identifying the category information of the returned items, the robotic arm can automatically sort the returned items on the assembly line.
[0090] Step 102, inputting the multiple images of the returned items into the trained image recognition model, thereby outputting the category information of the returned items; wherein the training process of the image recognition model includes: using the multiple images of the training items and the location information and category information of the training items marked in each image, performing supervised training on the image recognition model to update the model parameters, thereby obtaining a trained image recognition model.
[0091] Optionally, the image recognition module can be a yoloV5 convolutional network. Figure 3 As shown, the image recognition model includes a reinforcement layer (Mosaic), an anchor layer (anchor), a convolution layer (Conv), a regularization layer (Batch Normalization), an activation function layer (Silu), a feature pyramid network (FPN) + a spatial pyramid pooling network (SPP), and a head network (head); wherein the reinforcement layer is used to enhance the features of the input image, such as stretching, cropping, coloring, random filling, etc., the anchor layer is used to predict the anchor frame, the convolution layer is used to extract features from the image, the regularization layer is used to regularize the extracted features, and regularization can solve the problems of gradient disappearance and explosion, thereby accelerating model convergence, and the activation function layer introduces nonlinear factors to improve the model expression ability; the feature pyramid network + the spatial pyramid pooling network is used to fuse features, and the head network outputs prediction results.
[0092] Due to the characteristics of assembly line operations, the positions of reverse items on the assembly line are random. This is because the items are dumped onto the assembly line in batches at the entrance of the assembly line each time they are sorted. Therefore, the positions where the items may appear in the camera during actual detection are uncertain, and there is a high probability of occlusion between the items. In addition, reverse items cover multiple categories, and the shapes and volumes of the goods vary greatly. Therefore, the embodiment of the present invention uses an enhancement layer to perform feature enhancement processing on the data.
[0093] The output vector of the image recognition model is divided into three parts: one is the N-dimensional [0,1] vector used to determine the item label, that is, the category information; the other is the four-dimensional vector used to determine the item location; and the last part is the confidence of the prediction. Figure 3 As shown, the detection result of the first head network is a and the confidence is 0.6, the detection result of the second head network is b and the confidence is 0.6, and the detection result of the third head network is a and the confidence is 0.3. So, the confidence of the detection result a is (0.6+0.3) / 3=0.3, and the confidence of the detection result b is 0.3 / 3=0.1, so the final detection result output by the model is a.
[0094] Compared with other algorithms, yoloV5 has the following advantages: First, yoloV5 is based on the Pytorch framework, which is highly integrated and easy to encapsulate, which makes it more conducive to mass production and suitable for reverse supply chain operations; secondly, yoloV5 uses an adaptive anchor frame to capture images, which makes it excellent in detecting small objects. Since the reverse return supply line covers a variety of categories such as beauty, fresh food and electronic products, many of which are small in size, this advantage of yoloV5 makes it more suitable for reverse return sorting operations than other image detection algorithms; in addition, the yolo series of algorithms have good support for the capture and recognition of dynamic images, which makes it very suitable for the reverse return supply line sorting method.
[0095] Since the returned items from each return warehouse are quite different, in order to improve the model training speed and model recognition accuracy, an image recognition model can be trained for each return warehouse. When processing returned items, each return warehouse uses the trained image recognition model to output the category information of the returned items, such as secondary categories or tertiary categories.
[0096] Optionally, the training process of the image recognition model includes: constructing a data set, the data set including multiple images of training objects and location information and category information of the training objects marked in each image, and dividing the data set into a training set, a verification set and a test set; using the training set to perform supervised training on the image recognition model to update model parameters, thereby obtaining a trained image recognition model; using a portion of the data in the verification set to verify the trained image recognition model to update the model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera; and using the test set to test the trained image recognition model. First, determine the items handled by the return warehouse, obtain multiple images of the items handled by the return warehouse (i.e., training items) from the database, and mark the location information (such as the starting point coordinates, length, and width of the anchor box) and category information (such as secondary categories or tertiary categories) of the items in each image, thereby constructing a data set; then divide the data set into a training set, a validation set, and a test set according to a certain ratio (such as 1:2:1); then use the training set to perform supervised training on the image recognition model to update the model parameters, thereby obtaining a trained image recognition model; then use a portion of the data in the validation set to verify the trained image recognition model to update the model parameters, and use another portion of the data in the validation set to adjust the position and / or angle of each camera; finally, use the test set to test the trained image recognition model.
[0097] The first and second level classifications processed by each return warehouse generally do not fluctuate much, but the third level classifications often fluctuate over time. Therefore, the embodiment of the present invention marks the category information as the third level category; and the appearance and packaging of items in the same category will also change over time. Based on the above two situations, the image recognition model can be iterated at a certain time cycle, that is, retrained every certain period of time (such as two weeks, three weeks, four weeks or half a year); in each iteration, historical data within the past period of time is taken to form a training set and input into the image recognition model for learning. The historical data can be obtained from the bottom table of the scanned delivery note.
[0098] Optionally, constructing a data set includes: for each category, calculating the representative value of each item under the category, so as to screen out training items under the category; calculating the volume variance of each training item under each category, and sorting the categories in descending order of volume variance; obtaining images corresponding to the categories from an image database, so as to construct a data set; wherein, the higher the category is in the sorting, the more images corresponding to the category obtained from the image database are.
[0099] For example, in a return sorting scenario at a return warehouse, you only need to identify the third-level category to which the item belongs to complete the sorting. Therefore, you first need to count the category information in the historical data to ensure that each third-level category has at least one set of training data. For example, if the return warehouse handles three third-level categories of music boxes, desk lamps, keyboards, tea cups, and carpets in the historical data, then you need to ensure that each of these five third-level categories has training data.
[0100] In the embodiment of the present invention, the representative value of each item in each category is first calculated, and the training items with larger representative values in each category are screened out. Then, the volume variance of each training item in each category is calculated and sorted. Since the volume variance reflects the consistency of the volume of the items, the embodiment of the present invention appropriately increases the training data for the categories with poor consistency. For example, for the top 50% of the categories, 2,000 images of each item are selected for annotation to form a data set, and for the bottom 50% of the categories, 1,500 images of each item are selected for annotation to form a data set.
[0101] Optionally, the representative value of each item under the category is calculated to screen out training items under the category, including: for each item, calculating the average daily inventory outflow and unit volume of the item in the past period; calculating the maximum average daily inventory outflow, minimum average daily inventory outflow, maximum unit volume and minimum unit volume of the category in the past period; for each item, calculating the representative value of the item based on the average daily inventory outflow and unit volume of the item in the past period, the maximum average daily inventory outflow, minimum average daily inventory outflow, maximum unit volume and minimum unit volume of the items under the category in the past period; screening out items with representative values greater than or equal to the representative threshold as training items under the category.
[0102] Optionally, calculating the representative value of the item according to the average daily shipment quantity, the single piece volume of the item in the past period, the maximum average daily shipment quantity, the minimum average daily shipment quantity, the maximum single piece volume and the minimum single piece volume of the items under the category in the past period includes: calculating the representative value of the item using the following formula:
[0103]
[0104] Among them, P i is the representative value of item i, O i is the average daily outbound volume of item i in the past period of time, V i is the volume of a single item i, O max , O min 、V max 、V min , They are the maximum daily average outbound volume, minimum daily average outbound volume, maximum single piece volume, minimum single piece volume, and average single piece volume of the items in the category in the past period of time; a, b, and c are custom parameters. For example, a = 0.5, b = 0.6, and c = 0.4.
[0105] In the embodiment of the present invention, it is necessary to select Ki types of SKUs (i.e., training items) for each category i to construct a data set. The selection process of Ki is as follows:
[0106] Taking the third-level category as an example, the average daily outbound quantity O and the single unit volume V of each sku under the third-level category in the past period of time are calculated, and the maximum average daily outbound quantity O of the sku under the third-level category is obtained. max , minimum daily average outbound quantity O min , and the maximum single unit volume V of the sku under the third-level category max , Minimum single piece volume V min And the average unit volume of the SKU under the third-level category Use the above formula to calculate the representative value P of each SKU under the three-level category i For example, each sku is a group, and the sku with Pi greater than 0.5 is selected to form the data set of the third-level category, and the number of selected sku is Ki.
[0107] From the calculation formula of the representative value, it can be seen that if the average daily outbound volume of the sku is large, it is considered representative; if the single-piece volume of the sku is significantly different from the average single-piece volume of the third-level category, it is considered representative.
[0108] Step 103: sorting the returned items according to the category information of the returned items.
[0109] After identifying the category information of the returned items, a robotic arm or other automated equipment can be used to automatically sort the returned items, so as to improve the efficiency of sorting and reduce the investment in labor costs. Since the installation position of the camera and the angle of the camera relative to the assembly line are known, the position of the object to be detected in the image can be obtained after detection, and then the specific coordinates of the object on the assembly line can be obtained, so the returned items can be sorted by a robotic arm or other automated equipment.
[0110] like Figure 4As shown, the image of the returned item is input into the trained image recognition model, and the image recognition model outputs the category information of the returned item, and then the returned items are automatically sorted according to the category information of the returned items. After a period of time, the sorting results are sampled and checked to find out the returned items that were sorted incorrectly, and the image data of these returned items are summarized, and the location information and category information of the returned items are marked on each image, and the image recognition model is retrained with these data; after the image recognition model is trained, the returned items that were sorted incorrectly are resorted until there are no returned items that were sorted incorrectly in the sampling results.
[0111] According to the various embodiments described above, it can be seen that the embodiments of the present invention output the category information of the returned items by inputting multiple images of the returned items into the trained image recognition model, and the technical means of sorting the returned items according to the category information of the returned items solves the technical problems of low efficiency and high labor cost investment in the prior art for return sorting. The embodiments of the present invention identify the category information of the returned items through the image recognition model, improve the automation level of the reverse return sorting process, break the original operation mode of secondary sorting of problematic items by manual sorting, improve the efficiency of the return sorting process, and reduce the labor cost of reverse return sorting.
[0112] Figure 5 : is a flow chart of a return sorting method according to a reference embodiment of the present invention. As another embodiment of the present invention, Figure 5 As shown, the return sorting method may include:
[0113] Step 501: construct a data set, which includes multiple images of training objects and location information and category information of the training objects marked in each image, and divide the data set into a training set, a validation set, and a test set.
[0114] Step 502: Use the training set to perform supervised training on the image recognition model to update the model parameters, thereby obtaining a trained image recognition model.
[0115] Step 503: Use a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and use another portion of the data in the verification set to adjust the position and / or angle of each camera set on the assembly line.
[0116] by Figure 2For example, the assembly line is provided with cameras A, B and C, wherein camera A is located and fixed directly above the assembly line 10, so that camera A can capture images directly above the returned items, camera A is placed close to the robot arm and parallel to it, camera B and camera C are slidably connected to the first slide rail 20 and the second slide rail 30 respectively, the first slide rail 20 and the second slide rail 30 are symmetrically located on both sides above the assembly line 10, and camera B can move along the first slide rail 20, and camera C can move along the second slide rail 30. During training, the angles and positions of cameras B and C are adjustable, but they will be fixed once the model training is completed, and it is necessary to ensure that both cameras B and C can capture the situation 2 meters in front of the robot arm to ensure that the robot arm has enough reaction time.
[0117] The Yolo series of algorithms have the functions of dynamic tracking and recognition, so in theory only one camera is needed to identify the goods. However, considering the actual operation situation, since the sorting is ultimately performed by a robotic arm, the camera head needs to be close to the position of the robotic arm to provide the robotic arm with accurate item location information, which is the role of camera head A; in order to achieve the above purpose, camera head A needs to face the assembly line, which may cause occlusion problems in actual use. For example, if a watermelon and an apple are transported to A, then due to the large size of the watermelon, it is very likely to completely block the apple behind it from the front, thereby affecting the recognition and tracking of the apple (yolo can handle partial occlusion, but if an object is always completely blocked, it is difficult to identify it). On this basis, the embodiment of the present invention introduces cameras B and C, so that they shoot the assembly line at an angle from a distance on the side. These three cameras use the same trained model, so through the cooperation of the three of them, the detection blind spots caused by the mutual occlusion of items on the assembly line can be eliminated.
[0118] Optionally, step 503 may include: using a part of the data in the verification set to verify the trained image recognition model to update the model parameters, thereby obtaining the verification accuracy of the image recognition model; placing the training objects corresponding to another part of the data in the verification set on the assembly line, respectively capturing images of the training objects through the second camera and the third camera, and inputting the images into the trained image recognition model, thereby outputting category information of the training objects, and comparing the category information of the objects with another part of the data in the verification set to obtain the camera accuracy; dividing the camera accuracy by the verification accuracy to obtain a accuracy ratio; and adjusting the position and / or angle of the second camera and the third camera according to the accuracy ratio.
[0119] In an embodiment of the present invention, the position of camera A is fixed, and cameras B and C each select 5 to 10 initial positions according to the aspect ratio of the assembly line, and the initial angles of B and C are 45 degrees to the horizontal direction. After the image recognition model training is completed, 20% (or other values, which are not limited by the embodiment of the present invention) are randomly selected from the verification set to form a set T for determining the camera position, and the corresponding skus in the set T are placed on the assembly line. The detection accuracy Cp of the image recognition model is recorded for each position of camera B and camera C, and the position with the highest detection accuracy is taken as the installation position of camera B and camera C. After determining the installation position of camera B and camera C, adjust the angles of camera B and camera C, input set T directly into the image recognition model, record the model accuracy Ct, and divide the detection accuracy Cp corresponding to camera B and camera C by the model accuracy Ct, that is,
[0120] if (It may also be other values, which are not limited in the embodiment of the present invention), then the angle is used as the installation angle of camera B and camera C; if Then re-determine the installation position and installation angle of camera B and camera C until
[0121] Step 504: Use the test set to test the trained image recognition model.
[0122] Step 505 , collecting multiple images of the returned items by using cameras arranged at different positions on the production line.
[0123] Step 506: Input the multiple images of the returned items into the trained image recognition model, so as to output the category information of the returned items.
[0124] Step 507: sort the returned items according to the category information of the returned items.
[0125] In addition, the specific implementation content of the return sorting method in a reference embodiment of the present invention has been described in detail in the return sorting method described above, so the repeated content will not be described here.
[0126] Figure 6 Schematic diagram of a return sorting device according to an embodiment of the present invention. Figure 6As shown, the return sorting device 600 includes an acquisition module 601, an identification module 602 and a sorting module 603; wherein the acquisition module 601 is used to acquire multiple images of the return items through cameras arranged at different positions on the assembly line; the identification module 602 is used to input the multiple images of the return items into a trained image recognition model, thereby outputting the category information of the return items; wherein the training process of the image recognition model includes: using multiple images of the training items and the position information and category information of the training items marked in each image, the image recognition model is supervised trained to update the model parameters, thereby obtaining a trained image recognition model; the sorting module 603 is used to sort the return items for return according to the category information of the return items.
[0127] Optionally, a training module is also included, for:
[0128] Constructing a data set, the data set including multiple images of training objects and location information and category information of the training objects marked in each image, and dividing the data set into a training set, a validation set, and a test set;
[0129] Performing supervised training on the image recognition model using the training set to update model parameters, thereby obtaining a trained image recognition model;
[0130] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera;
[0131] The trained image recognition model is tested using the test set.
[0132] Optionally, the training module is further used to:
[0133] For each category, calculate the representative value of each item under the category, so as to screen out training items under the category;
[0134] Calculate the volume variance of each training item in each category, and sort the categories in descending order of volume variance;
[0135] Images corresponding to the various categories are obtained from an image database, thereby constructing a data set; wherein, the higher the category is in the ranking, the more images corresponding to the category are obtained from the image database.
[0136] Optionally, the training module is further used to:
[0137] For each item, calculate the average daily outbound quantity and unit volume of the item in the past period of time;
[0138] Calculate the maximum average daily outbound volume, minimum average daily outbound volume, maximum single piece volume, and minimum single piece volume of the category in the past period of time;
[0139] For each item, calculate the representative value of the item based on the average daily shipment volume, unit volume of the item in the past period, the maximum average daily shipment volume, minimum average daily shipment volume, maximum unit volume, and minimum unit volume of the items in the category in the past period;
[0140] Items with representative values greater than or equal to a representative threshold are selected as training items under the category.
[0141] Optionally, the training module is further used to:
[0142] The representative value of the item is calculated using the following formula:
[0143]
[0144] Among them, P i is the representative value of item i, O i is the average daily outbound volume of item i in the past period of time, V i is the volume of a single item i, O max , O min 、V max 、V min , They are the maximum daily average outbound quantity, minimum daily average outbound quantity, maximum single piece volume, minimum single piece volume and average single piece volume of the items in the category in the past period of time; a, b, c are custom parameters.
[0145] Optionally, the camera includes a first camera, a second camera and a third camera, the first camera is fixed directly above the assembly line, the second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail; the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line.
[0146] Optionally, the training module is further used to:
[0147] Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, thereby obtaining a verification accuracy rate of the image recognition model;
[0148] The training objects corresponding to the other part of the data in the verification set are placed on the production line, and images of the training objects are respectively captured by the second camera and the third camera, and the images are input into the trained image recognition model, thereby outputting category information of the training objects, and comparing the category information of the objects with the other part of the data in the verification set, thereby obtaining the camera accuracy;
[0149] Divide the camera accuracy by the inspection accuracy to obtain an accuracy ratio;
[0150] The positions and / or angles of the second camera and the third camera are adjusted according to the accuracy ratio.
[0151] It should be noted that the specific implementation content of the return sorting device of the present invention has been described in detail in the return sorting method described above, so the repeated content will not be described here.
[0152] An embodiment of the present invention also provides a return sorting system, comprising a return sorting device as described in the above embodiment and cameras arranged at different positions on the assembly line. Optionally, the return sorting system also includes a first slide rail and a second slide rail, the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line; the cameras include a first camera, a second camera and a third camera, the first camera is fixed directly above the assembly line, the second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail. Optionally, the return sorting system also includes: a robotic arm, which is used to automatically return and sort the returned items according to the category information of the returned items.
[0153] Figure 7 An exemplary system architecture 700 is shown to which the return sorting method or return sorting device according to the embodiment of the present invention can be applied.
[0154] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0155] Users can use terminal devices 701, 702, and 703 to interact with server 705 through network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).
[0156] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.
[0157] Server 705 may be a server that provides various services, such as a backend management server (for example only) that provides support for shopping websites browsed by users using terminal devices 701, 702, and 703. The backend management server may analyze and process the received data such as item information query requests, and feed back the processing results to the terminal device.
[0158] It should be noted that the return sorting method provided in the embodiment of the present invention is generally executed by the server 705, and accordingly, the return sorting device is generally set in the server 705. The return sorting method provided in the embodiment of the present invention can also be executed by the terminal devices 701, 702, and 703, and accordingly, the return sorting device can be set in the terminal devices 701, 702, and 703.
[0159] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0160] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0161] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0162] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.
[0163] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.
[0164] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0165] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0166] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a collection module, a recognition module and a sorting module, wherein the names of these modules do not constitute a limitation on the modules themselves in some cases.
[0167] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device implements the following method: multiple images of returned items are collected by cameras arranged at different positions on the assembly line; the multiple images of the returned items are input into a trained image recognition model, thereby outputting the category information of the returned items; wherein the training process of the image recognition model includes: using multiple images of training items and the location information and category information of the training items marked in each image, the image recognition model is supervisedly trained to update the model parameters, thereby obtaining a trained image recognition model; and the returned items are sorted for return according to the category information of the returned items.
[0168] As another aspect, an embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in any of the above embodiments is implemented.
[0169] According to the technical solution of the embodiment of the present invention, because multiple images of returned items are input into a trained image recognition model to output the category information of the returned items, and the returned items are sorted according to the category information of the returned items, the technical problems of low return sorting efficiency and large labor cost investment in the prior art are overcome. The embodiment of the present invention identifies the category information of returned items through an image recognition model, improves the automation level of the reverse return sorting process, breaks the original operation mode of secondary sorting of problematic items through manual sorting, improves the efficiency of the return sorting process, and reduces the labor cost of reverse return sorting.
[0170] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A return sorting method, characterized in that: include: Collect multiple images of returned items through cameras set at different positions on the production line; Inputting the multiple images of the returned items into a trained image recognition model, thereby outputting the category information of the returned items; wherein the training process of the image recognition model includes: using the multiple images of the training items and the position information and category information of the training items marked in each image, performing supervised training on the image recognition model to update the model parameters, thereby obtaining a trained image recognition model; The returned items are sorted for return according to the category information of the returned items.
2. The method according to claim 1, characterized in that The training process of the image recognition model includes: Constructing a data set, the data set including multiple images of training objects and location information and category information of the training objects marked in each image, and dividing the data set into a training set, a validation set, and a test set; Performing supervised training on the image recognition model using the training set to update model parameters, thereby obtaining a trained image recognition model; Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera; The trained image recognition model is tested using the test set.
3. The method according to claim 2, characterized in that Build a dataset, including: For each category, calculate the representative value of each item under the category, so as to screen out training items under the category; Calculate the volume variance of each training item in each category, and sort the categories in descending order of volume variance; Images corresponding to the various categories are obtained from an image database, thereby constructing a data set; wherein, the higher the category is in the ranking, the more images corresponding to the category are obtained from the image database.
4. The method according to claim 3, characterized in that Calculate the representative value of each item under the category, so as to screen out training items under the category, including: For each item, calculate the average daily outbound quantity and unit volume of the item in the past period of time; Calculate the maximum average daily outbound volume, minimum average daily outbound volume, maximum single piece volume, and minimum single piece volume of the category in the past period of time; For each item, calculate the representative value of the item based on the average daily shipment volume, unit volume of the item in the past period, the maximum average daily shipment volume, minimum average daily shipment volume, maximum unit volume, and minimum unit volume of the items in the category in the past period; Items with representative values greater than or equal to a representative threshold are selected as training items under the category.
5. The method according to claim 4, characterized in that The representative value of the item is calculated based on the average daily shipment volume, unit volume of the item in the past period, the maximum average daily shipment volume, minimum average daily shipment volume, maximum unit volume and minimum unit volume of the items under the category in the past period, including: The representative value of the item is calculated using the following formula: Among them, P i is the representative value of item i, O i is the average daily outbound volume of item i in the past period of time, V i is the volume of a single item i, O max , O min 、V max 、V min , They are the maximum daily average outbound quantity, minimum daily average outbound quantity, maximum single piece volume, minimum single piece volume and average single piece volume of the items in the category in the past period of time; a, b, c are custom parameters.
6. The method according to claim 1, characterized in that The cameras include a first camera, a second camera and a third camera. The first camera is fixed directly above the assembly line. The second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail. The first slide rail and the second slide rail are symmetrically located on both sides above the assembly line.
7. The method according to claim 6, characterized in that Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, and using another portion of the data in the verification set to adjust the position and / or angle of the camera, including: Using a portion of the data in the verification set to verify the trained image recognition model to update model parameters, thereby obtaining a verification accuracy rate of the image recognition model; The training objects corresponding to the other part of the data in the verification set are placed on the production line, and images of the training objects are respectively captured by the second camera and the third camera, and the images are input into the trained image recognition model, thereby outputting category information of the training objects, and comparing the category information of the objects with the other part of the data in the verification set, thereby obtaining the camera accuracy; Divide the camera accuracy by the inspection accuracy to obtain an accuracy ratio; The positions and / or angles of the second camera and the third camera are adjusted according to the accuracy ratio.
8. A return sorting device, characterized in that: include: A collection module, used to collect multiple images of returned items through cameras arranged at different positions on the assembly line; A recognition module, used to input the multiple images of the returned items into a trained image recognition model, so as to output the category information of the returned items; wherein the training process of the image recognition model includes: using the multiple images of the training items and the position information and category information of the training items marked in each image, performing supervised training on the image recognition model to update the model parameters, so as to obtain a trained image recognition model; The sorting module is used to sort the returned items according to the category information of the returned items.
9. A return sorting system, characterized in that: It comprises the return sorting device as claimed in claim 8 and cameras arranged at different positions on the assembly line.
10. The system according to claim 9, characterized in that It also includes a first slide rail and a second slide rail, wherein the first slide rail and the second slide rail are symmetrically located on both sides above the assembly line; The cameras include a first camera, a second camera and a third camera. The first camera is fixed directly above the assembly line. The second camera and the third camera are slidably connected to the first slide rail and the second slide rail respectively, and can move along the first slide rail and the second slide rail.
11. The system according to claim 9, characterized in that Also includes: The robot arm is used to automatically sort the returned items according to the category information of the returned items.
12. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
13. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Grading sorting conveyor belt
CN120115412A
A grading and sorting conveyor belt
CN120115412B
