Sorting object allocation method and device

Abstract

The invention discloses a sorting object allocation method and device, and relates to the technical field of warehouse logistics. One specific embodiment of the method comprises the following steps: acquiring an image set corresponding to a sorting grid, wherein the image set comprises one or more grid images corresponding to the sorting grid; target areas corresponding to sorting objects are recognized from the one or more grid images respectively; according to at least one item of the number of target areas in each grid image and the positions of the target areas in the grid images, the current number of sorting objects corresponding to the sorting grids is determined; and allocating the sorting objects according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grids. According to the embodiment, the sorting objects can be accurately and reasonably allocated, the workload of each sorting object can be balanced, and the sorting efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of warehousing logistics, and in particular to a sorting object allocation method and device. Background Art

[0002] During the sorting process at the logistics sorting point, the sorting quantities of the sorting processes corresponding to different shifts are generally different. Therefore, for the sorting processes of different shifts, the sorting objects (such as sorting robots or sorting employees, etc.) need to be allocated according to the sorting quantities.

[0003] At present, the manual allocation method is generally adopted, that is, the sorting team leader allocates the sorting objects according to the changes in the sorting volume. However, since the changes in the sorting volume of different batches are relatively random, when the manual allocation is adopted, the sorting team leader cannot make reasonable and accurate allocations in the first time, which leads to low sorting efficiency and easily increases the workload of the sorting objects. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for allocating sorting objects, which can identify the target area corresponding to the sorting object from the image set corresponding to the acquired sorting grid, and then determine the current number of sorting objects corresponding to the sorting grid according to the number and / or position of the target area, and then allocate the sorting objects according to the current number of sorting objects in the sorting grid and the current sorting quantity. In this way, the sorting objects in the sorting grid can be automatically and accurately identified, and then the sorting objects can be accurately and reasonably allocated according to the identification results, so that the allocation of the sorting objects can adapt to the changes in the sorting quantity, which is conducive to balancing the workload of each sorting object and improving the sorting efficiency.

[0005] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for allocating sorting objects is provided.

[0006] A method for allocating sorting objects according to an embodiment of the present invention comprises: acquiring an image set corresponding to a sorting grid, wherein the image set comprises one or more grid images corresponding to the sorting grid;

[0007] Identifying target areas corresponding to the sorting objects from one or more of the grid images respectively;

[0008] Determine the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each of the grid images and the position of the target area in the grid image;

[0009] The sorting objects are allocated according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid.

[0010] Optionally, the step of respectively identifying target areas corresponding to the sorting objects from one or more grid images includes:

[0011] Inputting the grid image into a detection model based on a convolutional neural network, wherein the detection model is trained based on logistics images annotated with logistics objects and object labels;

[0012] Determine one or more object regions, location information and confidence of the object regions according to the output of the detection model;

[0013] According to at least one of the position information and the confidence level of each of the object regions, a target region corresponding to the sorting object in the grid image is determined.

[0014] Optionally, determining a target area corresponding to the sorting object in the grid image according to at least one of the position information and the confidence of each of the object areas includes:

[0015] For each of the grid images: determining the center point of the object area and the angular coordinates of the object area according to the position information;

[0016] Determine whether the object area is located in the sorting area corresponding to the sorting grid according to at least one of the following: whether the center point of the target area is located in the sorting area, and whether the number of target angular coordinates of the target area is not less than a preset number threshold; wherein the target angular coordinate is an angular coordinate located in the sorting area;

[0017] In the case where it is determined that the object area is located within the sorting area, the object area is determined to be a target area corresponding to the sorting object, and the number of target areas in the grid image is incremented.

[0018] Optionally, determining the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each of the grid images and the positions of the target areas in the grid images includes:

[0019] Determining the mode of the target areas according to the number of target areas in each of the grid images in the image set;

[0020] The current number of sorting objects corresponding to the sorting grid is determined according to the majority.

[0021] Optionally, when there are multiple modes, determining the current number of sorting objects corresponding to the sorting grid according to the modes includes:

[0022] Determine the grid image corresponding to each mode, and calculate the confidence mean of the grid image; take the grid image with the largest confidence mean as the target grid image in the image set, and determine the current number of the sorted objects according to the object area included in the target grid image.

[0023] Optionally, the maximum number is taken as the current number of the sorting objects.

[0024] Optionally, acquiring an image set corresponding to a sorting grid includes:

[0025] Determine the camera device corresponding to the sorting grid, and obtain the video stream captured by the camera device;

[0026] Extract frames from the video stream according to a preset frame extraction interval to obtain the grid image;

[0027] The sorting area corresponding to the sorting grid is marked in the grid image, and the image set corresponding to the sorting grid is obtained according to the marking result.

[0028] Optionally, when the sorting grid corresponds to a plurality of camera devices, the step of obtaining the image set corresponding to the sorting grid includes:

[0029] Extract frames from the video streams collected by the multiple camera devices respectively according to the same frame extraction interval to obtain the grid image;

[0030] For a group of grid images corresponding to the same frame extraction interval: marking the boundary areas corresponding to the sorting area and the plurality of camera devices in the grid images;

[0031] An image set corresponding to the sorting grid is obtained according to the labeling result.

[0032] Optionally, determining the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each of the grid images and the positions of the target areas in the grid images includes:

[0033] Determine a first grid image and a second grid image that intersects with the first grid image from each group of the grid images;

[0034] Determine a first target area in a boundary area of ​​the first grid image according to a position of the target area in the grid image, and determine a second target area in a boundary area of ​​the second grid image;

[0035] determining a relative distance between the first target area and the second target area;

[0036] According to the relative distance, it is determined whether the first target area and the second target area correspond to the same sorting object.

[0037] Optionally, determining the relative distance between the first target area and the second target area includes:

[0038] Determine a first position coefficient of the first target area relative to a boundary area of ​​the first grid image, and a second position coefficient of the second target area relative to a boundary area of ​​the second grid image;

[0039] The relative distance is determined according to a difference between the first position coefficient and the second position coefficient.

[0040] Optionally, the determining, according to the relative distance, whether the first target area and the second target area correspond to the same sorting object further includes:

[0041] For the first target area and the second target area whose relative distance is less than the preset distance threshold:

[0042] calculating a similarity between the first target area and the second target area;

[0043] It is determined whether the first target area and the second target area correspond to the same sorting object according to the similarity.

[0044] Optionally, the calculating the similarity between the first target area and the second target area includes:

[0045] Constructing a target area set according to the first target area and a second object area whose relative distance to the first target area is less than a preset distance threshold;

[0046] Extracting feature vectors of each target object region in the target region set using a feature extraction model;

[0047] The cosine similarity between the first target region and the second target region in the target region set is calculated according to the feature vector.

[0048] Optionally, for a multi-frame grid image corresponding to any camera: determining a mode of target areas in the multi-frame grid image, and a mode frame corresponding to the mode;

[0049] Determining a target frame with a maximum confidence mean from the majority frames according to the confidence mean corresponding to the target area of ​​the majority frames;

[0050] The target frame is used as the first grid image.

[0051] Optionally, determining the current number of sorting objects corresponding to the sorting grid includes:

[0052] Determine a first number of target areas corresponding to sorting areas and boundary areas in the first grid image and the second grid image;

[0053] Determine a second quantity corresponding to the same sorting object in the boundary area;

[0054] The difference between the first quantity and the second quantity is used as the current quantity of the sorting objects corresponding to the sorting grid.

[0055] Optionally, allocating the sorting objects according to the number of the sorting objects and the current sorting amount corresponding to the sorting grid includes:

[0056] Determining the target number of sorting objects required for the sorting grid according to the ratio of the current sorting volume to the standard workload;

[0057] In the case where the difference between the target quantity and the current quantity is greater than a preset first threshold, calling in sorting objects for the sorting grid;

[0058] When the difference between the target quantity and the current quantity is less than a preset second threshold, a call-out object is determined from the sorting grid, and the call-out object is allocated to other sorting grids.

[0059] Optionally, after allocating the sorting objects according to the current number of the sorting objects and the current sorting amount corresponding to the sorting grid, the method further includes:

[0060] The workload and working time of each sorting object are recorded.

[0061] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for allocating sorting objects is provided.

[0062] A sorting object allocation device according to an embodiment of the present invention comprises: an image acquisition module, an area determination module, a quantity determination module and an allocation module; wherein:

[0063] The image acquisition module is used to acquire an image set corresponding to a sorting grid, wherein the image set includes one or more grid images corresponding to the sorting grid;

[0064] The area determination module is used to respectively identify target areas corresponding to the sorting objects from one or more of the grid images;

[0065] The quantity determination module is used to determine the current quantity of sorting objects corresponding to the sorting grid according to at least one of the quantity of target areas in each of the grid images and the position of the target areas in the grid images;

[0066] The allocation module is used to allocate the sorting objects according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid.

[0067] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device for allocating and sorting objects is provided.

[0068] An electronic device for allocating and sorting objects according to an embodiment of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a method for allocating sorting objects according to an embodiment of the present invention.

[0069] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided.

[0070] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a method for allocating sorting objects according to an embodiment of the present invention is implemented.

[0071] One embodiment of the above invention has the following advantages or beneficial effects: it is possible to identify the target area corresponding to the sorting object from the acquired image set corresponding to the sorting grid, and then determine the current number of sorting objects corresponding to the sorting grid according to the number and / or position of the target area, and then allocate the sorting objects according to the current number of sorting objects in the sorting grid and the current sorting quantity. In this way, the sorting objects in the sorting grid are automatically and accurately identified, and then the sorting objects are accurately and reasonably allocated according to the identification results, so that the allocation of the sorting objects can adapt to the changes in the sorting quantity, which is conducive to balancing the workload of each sorting object and improving the sorting efficiency.

[0072] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0074] Figure 1 is a schematic diagram of the main steps of a method for allocating sorting objects according to an embodiment of the present invention;

[0075] Figure 2is a schematic diagram of a sorting area and a boundary area according to an embodiment of the present invention;

[0076] Figure 3 is a schematic diagram of main steps of another method for allocating sorting objects according to an embodiment of the present invention;

[0077] Figure 4 is a schematic diagram of main steps of another method for allocating sorting objects according to an embodiment of the present invention;

[0078] Figure 5 is a schematic diagram of main modules of a device for allocating sorting objects according to an embodiment of the present invention;

[0079] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0080] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0082] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0083] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0084] Figure 1 It is a schematic diagram of the main steps of a method for allocating sorting objects according to an embodiment of the present invention.

[0085] like Figure 1 As shown, a method for allocating sorting objects according to an embodiment of the present invention mainly includes the following steps S101 to S104:

[0086] Step S101: Acquire an image set corresponding to a sorting grid, wherein the image set includes one or more grid images corresponding to the sorting grid.

[0087] When it is necessary to allocate sorting objects, for example, when receiving a new batch of sorting tasks, the manager of the sorting site can retrieve the images captured by the surveillance camera of the sorting site, thereby obtaining the grid image set corresponding to the sorting task. Specifically, in an embodiment of the present invention, the camera device corresponding to the sorting grid can be determined, and the video stream captured by the camera device can be obtained; then, according to a preset frame extraction interval, the video stream is frame extracted to obtain the grid image; the sorting area corresponding to the sorting grid is marked in the grid image, and the image set corresponding to the sorting grid is obtained according to the marking result.

[0088] For example, after the manager of the sorting site determines the camera equipment of the sorting grid, the corresponding frame extraction algorithm is triggered, so that the frame extraction algorithm is programmed to perform the fixed time interval T freq (15min) Extract the video stream collected by the camera device. In other words, the video stream is pulled every 15 minutes through the frame extraction algorithm, and then the video stream is extracted. The frame extraction interval is T interval (such as 400ms), for example, a 2s video in the pulled video stream can be randomly selected, 5 frames can be extracted from it as a grid image, and the sorting area corresponding to the sorting grid is marked in the grid image to obtain the image set F = {frame0, frame1, frame2, frame3, frame4} corresponding to the sorting grid.

[0089] If the spatial range of the sorting grid is large and requires multiple cameras to monitor, then when acquiring the image set corresponding to the sorting grid, the images captured by the multiple cameras can be combined so that the combined grid image covers the entire sorting chute of the sorting grid. Specifically, in an embodiment of the present invention, after acquiring the video streams of multiple cameras, the video streams captured by the multiple cameras can be framed according to the same frame extraction interval to obtain the grid image; for a group of grid images corresponding to the same frame extraction interval: the sorting area and the boundary area corresponding to the multiple cameras are marked in the grid image; and the image set corresponding to the sorting grid is obtained according to the marking result.

[0090] For example, when two cameras (Camera A and Camera B) monitor the same sorting grid, the frame extraction algorithm still pulls the video stream collected by each camera every 15 minutes, and then extracts the video stream according to the same frame extraction interval T. interval =400ms to extract frames from the video stream. For example, 5 frames are extracted every 2s as grid images, and F is obtained. A ={frame A0,frame A1 ,frame A2 ,frame A3 ,frame A4} and F B ={frame B0 ,frame B1 ,frame B2 ,frame B3 ,frame B4 Then for a group of grid images corresponding to the same frame extraction interval (such as frame A0 and frame B0 ), in which the sorting area and the boundary area are marked, such as Figure 2 As shown, the diagonal filled part is the frame A0 and frame B0 The combination covers the entire sorting chute of the sorting grid, and the diagonal filling part is frame A0 and frame B0 The boundary area of ​​​​the frame A0 and frame B0 Corresponding sorting area, the remaining images of the extracted frame are compared with frame A0 and frame B0 By similar combination and annotation, we can obtain the image set corresponding to the sorting grid.

[0091] Step S102: identifying target areas corresponding to the sorting objects from one or more grid images respectively.

[0092] Identifying the target area from the grid image can be performed based on a detection model. In an embodiment of the present invention, the grid image can be input into a detection model based on a convolutional neural network, and the detection model is trained based on logistics images marked with logistics objects and object labels; one or more object areas, location information and confidence of the object areas are determined according to the output of the detection model; and the target area corresponding to the sorting object in the grid image is determined according to at least one of the location information and confidence of each of the object areas.

[0093] Among them, the detection model can use the YOLOv7 detector based on convolutional neural network, which is trained by data annotated with the position frame and category label (such as whether it is a sorting object) of the logistics scene personnel (logistics object). When identifying the target area, the grid image obtained by extracting the frame is input into the detector, and the detector can output the rectangular position information and confidence value (confidence is the probability of 0 to 1) of the object area in the picture, where the position information includes the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of the rectangle.

[0094] Then, the target area corresponding to the sorting object is screened out from the object area according to the position information and the confidence. In one embodiment of the present invention, whether the object area is the target area corresponding to the sorting grid can be determined according to whether the object area is located in the sorting area corresponding to the sorting grid. Specifically, whether the object area is located in the sorting area corresponding to the sorting grid can be determined according to whether the center point of the target area is located in the sorting area and whether the number of target angle coordinates of the target area is not less than a preset number threshold; wherein the target angle coordinates are the angle coordinates located in the sorting area. In the case where it is determined that the object area is located in the sorting area, the object area is determined to be the target area corresponding to the sorting object of the sorting grid, and then the number of target areas in the corresponding grid image is incremented. For example, when the position information of the rectangular frame corresponding to the object area includes its upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2), the coordinates of its center point can be determined according to the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2). Then, according to the coordinates of the center point and the angular coordinates of the sorting area in the coordinate system, it can be determined whether the coordinates of the center point are located in the sorting area. Similarly, according to the upper left corner coordinates and the lower right corner coordinates of the object area, the other two angular coordinates (x1, y2) and (x2, y1) of the rectangular frame corresponding to the object area can be determined. Then, according to the angular coordinates of the object area and the angular coordinates of the sorting area in the coordinate system, the number of target angular coordinates of the object area in the sorting area can be determined, and then according to the relationship between the number of target angular coordinates and the preset number threshold, it can be determined whether the object area is located in the sorting grid. For example, the preset number threshold is 2, that is, when two or more of the four angular coordinates of the object area are located in the sorting area corresponding to the sorting grid, it can be determined that the object area is located in the sorting area, and the object area can be used as the target area corresponding to the sorting personnel in the sorting grid, and the number of target areas in the corresponding grid image can be increased by 1.

[0095] In another embodiment of the present invention, it is also possible to determine whether the object area is a target area based on the confidence of the object area, such as determining the target area corresponding to the sorting object based on the object area whose confidence is greater than a preset confidence threshold. For example, the object area with a confidence greater than 0.5 is determined as the target area corresponding to the sorting object. In a preferred embodiment of the present invention, the confidence and the position are combined to determine the target area corresponding to the sorting object from the object area. It can be understood that for each grid image in the image set, the above method can be used to identify the target area corresponding to the sorting object from the grid image.

[0096] Step S103: determining the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each of the grid images and the position of the target area in the grid image.

[0097] After determining the target area corresponding to each grid image in the image set, the mode of the target areas corresponding to various grid images can be determined according to the number of target areas in each grid image, and then the current number of sorting objects corresponding to the sorting grid can be determined according to the mode.

[0098] Still taking the image set F={frame0, frame1, frame2, frame3, frame4} corresponding to the sorting grid as an example, if the number of target areas in frame0 is 3, the number of target areas in frame1 is 3, the number of target areas in frame2 is 2, the number of target areas in frame3 is 4, and the number of target areas in frame4 is 5, then the number set of target areas corresponding to the picture set is N={3, 3, 2, 4, 5}, then the mode of the target areas is 3, and it can be determined that the current number of sorting objects corresponding to the sorting grid is 3.

[0099] If there are multiple modes corresponding to each grid image, the largest mode can be used as the current number of sorting objects. For example, if the number of target areas in frame3 in the above example is 2, then the number of target areas corresponding to the picture set is N = {3, 3, 2, 2, 5}, at this time 2 and 3 are both the modes of the target areas corresponding to each grid image, and the larger mode 3 can be used as the current number of sorting objects. For another example, if the number of target areas in frame0 is 1, the number of target areas in frame1 is 2, the number of target areas in frame2 is 3, the number of target areas in frame3 is 4, and the number of target areas in frame4 is 5, the number of target areas corresponding to the picture set is N = {1, 2, 3, 4, 5}, then the mode 5 is used as the current number of sorting objects.

[0100] In another embodiment of the present invention, when there are multiple modes, the current number of sorting objects can be determined in combination with the confidence. Specifically, the grid image corresponding to each mode can be determined, and the confidence mean of the grid image can be calculated; the grid image with the largest confidence mean is used as the target grid image in the image set, and the current number of sorting objects is determined according to the object area included in the target grid image.

[0101] Still taking the above target area number set N = {3, 3, 2, 2, 5} as an example, the grid images corresponding to the majority (hereinafter referred to as the majority frame) are frame0, frame1, frame2 and frame3. According to the confidence corresponding to each target area in each majority frame, the confidence mean corresponding to the majority frame can be calculated. For example, there are two target areas in frame0, and the confidences of the two target areas are 0.8 and 0.9 respectively, then the confidence mean of frame0 is 0.85. Similarly, the confidence mean of other majority frames can be calculated (such as calculating the confidence mean corresponding to frame1, frame2 and frame3 as 0.81, 0.8, and 0.83 respectively). Then, according to the confidence mean corresponding to each majority frame, the majority frame with the largest confidence mean is used as the target grid image corresponding to the image set. In this example, frame0 is used as the target grid image, and the number of target areas in frame0 is used as the current number of sorting objects. In addition, the target grid image in the image set (hereinafter also referred to as the best frame) can be displayed on the front page for the administrator to retrieve and view.

[0102] In addition, in one embodiment of the present invention, in the case where the same sorting grid corresponds to multiple camera devices, the current number of sorting objects can be determined based on a group of grid images with an intersection area. Specifically, a first grid image and a second grid image that intersects with the first grid image can be determined from each group of the grid images; based on the position of the target area in the grid images, a first target area is determined in the intersection area of ​​the first grid image, and a second target area is determined in the intersection area of ​​the second grid image; then the relative distance between the first target area and the second target area is determined; and based on the relative distance, it is determined whether the first target area and the second target area correspond to the same sorting object.

[0103] In order to reduce the amount of data processing, the best frame can be used as the first grid image, that is, the current number of sorting objects is determined according to the best frame. The process of determining the best frame is similar to the process of determining the best frame in the above embodiment, that is, for the multi-frame grid images corresponding to each camera: determine the mode of the target area in the multi-frame grid image and the mode frame corresponding to the mode; then determine the target frame (best frame) with the largest confidence mean from the mode frame according to the confidence mean corresponding to the target area of ​​the mode frame. The following takes the best frame as the first grid image as an example to illustrate the process of determining the current number of sorting objects when the sorting grid corresponds to multiple camera devices. It can be understood that although the following embodiment is described by taking the best frame as an example, in the embodiment of the present invention, any grid image can be used as the first grid image, and then after determining whether the first target area and the second target area in a group of grid images correspond to the same sorting object, the current number of sorting objects is determined according to the number corresponding to the same sorting object. It is also possible to determine the current number of sorting objects by using the above-mentioned mode and / or confidence mean after determining the number of sorting objects corresponding to multiple groups of grid images.

[0104] The following takes the best frame as the first grid image as an example to specifically describe the process of determining the current number of sorting objects. Figure 2 , if the best frame is frame A0 , and the second grid image with the boundary region (hereinafter referred to as ROI) is frame B0 According to the frame A0 and frame B0 The positions of the target areas in the grid image are determined, and the first target area corresponding to the first grid image and the second target area corresponding to the second grid image are determined in the ROI. In other words, the first target area is the target area in the first grid image that exists in the boundary area, and the second target area is the target area in the second grid image that exists in the boundary area.

[0105] For example, if the square box corresponding to the target area overlaps with the boundary area, it can be determined that the target area exists in the boundary area. Figure 2 A_Box shown ROI and B_Box ROI ;frame A0 Other target areas that do not overlap with the boundary area can be marked as A_Box other ,frame B0 Other target areas that do not overlap with the boundary area can be marked as B_Box other Then, the number of target areas corresponding to the same sorting object can be determined according to the positions and numbers of the first target area and the second target area in the boundary area.

[0106] In other words, after getting the best frame A_frame extracted by camera A i After that, the number of target areas NA and their corresponding detection box set A_Box can be determined, and the detection box A_Box in A_Box that intersects with the boundary area A_ROI area can be determined ROI , whose number is NA_ROI, and the detection boxes that have no intersection with the A_ROI area are represented as the set A_Box other , the number is NA_other, then NA=NA_ROI+NA_other. Correspondingly, with the best frame A_frame i B_frame at the same index position i As the second grid image, we get B_frame accordingly i The number of target areas in the target area is NB, the corresponding detection box set is B_Box, and the detection box B_Box in B_Box that intersects with the boundary area B_ROI is determined ROI , whose number is NB_ROI, and the detection box that has no intersection with the boundary area B_ROI area is represented by the set B_Box other , the number is NB_other, then NB=NB_ROI+NB_other.

[0107] Then according to the first target area A_Box in the junction area ROI and the second target area B_Box ROI The relative distance between the first target area and the second target area is determined to determine whether the first target area and the second target area correspond to the same sorting object. In one embodiment of the present invention, a first position coefficient of the first target area relative to the boundary area of ​​the first grid image and a second position coefficient of the second target area relative to the boundary area of ​​the second grid image may be determined first; and then the relative distance is determined according to the difference between the first position coefficient and the second position coefficient.

[0108] For example, A_Box can be calculated as follows ROI The relative position coefficient of the detection box (first target area) relative to the A_ROI area: center_a / (ROI_a2-ROI_a1), where center_a is A_Box ROI The horizontal coordinate of the center point of the detection box, ROI_a1 is the horizontal coordinate of the left boundary of the A_ROI area, and ROI_a2 is the horizontal coordinate of the right boundary of the A_ROI area. In addition, the following calculation method is used to calculate B_Box ROI The relative position coefficient of the detection box (second target area) relative to the B_ROI area is: 1-center_b / (ROI_b2-ROI_b1), where center_b is B_BoxROI The abscissa of the center point of the detection box, ROI_b1 is the abscissa of the left boundary of the B_ROI area, and ROI_b2 is the abscissa of the right boundary of the B_ROI area. Thus, A_Box is obtained. ROI Detection box, B_Box ROI The relative position coefficients of the detection boxes and their respective ROI areas. The larger the relative position coefficient, the closer the target area corresponding to the A camera is to the right side area of the field of view, and the closer the target area corresponding to the B camera is to the left side area of the field of view. At the same time, ROI Detection box and B_Box ROI The smaller the absolute value of the difference in the position coefficients of the detection boxes and B_Box, the closer their actual physical positions are. Set the distance threshold Loc_Thr. When |A_Index - B_Index| < Loc_Thr, it is determined that ROI Detection box and B_Box ROI The detection boxes match, that is, they may be the same person. Among them, A_Index is the relative position coefficient of ROI the A_Box detection box, and B_Index is the relative position coefficient of ROI the B_Box detection box. Loc_Thr is generally taken as 0.5. Thus, the set of detection boxes with position matching {A_B i , i = 0,..., n0} is obtained, where n0 is the total number of matches. Then, the target areas corresponding to the same sorting object can be determined according to this set of detection boxes with position matching. For example, when NA_ROI is 3 and NB_ROI is also 3, that is, when the first grid image and the second grid image respectively include 3 first target areas and second target areas that intersect with the boundary area, the relative distances of these first target areas and second target areas are as shown in Table 1 below:

[0109] Table 1

[0110]

[0111] When the distance threshold Loc_Thr is 0.5, the set of detection boxes with position matching A_B ROI0 corresponding to A_Box is i＝ {A_B0, A_B1, A_B2}, and the set of detection boxes with position matching A_B corresponding to A_BoxROI1 is i＝ {A_B1, A_B1, A_B2}, and the set of detection boxes with position matching A_B ROI2 corresponding to A_Box is i＝ {A_B0, A_B2}. In other words, the set of detection boxes with position matching corresponding to the first grid image and the second grid image is {A0_B0, A0_B1, A0_B2, A1_B0, A1_B1, A1_B2, A2_B0, A2_B2}.

[0112] In one embodiment of the present invention, in order to further improve the recognition accuracy, for the above-mentioned position matching of the first target area and the second area, that is, for the first target area and the second area whose relative distance is less than the preset distance threshold, the similarity between the two can be calculated, and further based on the similarity, it can be determined whether the two correspond to the same sorting object. Specifically, a target area set can be constructed based on the first target area and the second object area whose relative distance to the first target area is less than the preset distance threshold; then a feature extraction model is used to extract the feature vectors of each target object area in the target area set; based on the feature vectors, the cosine similarity between the first target area and the second target area in the target area set is calculated, thereby determining the similarity between the first target area and the second target area.

[0113] The target area set can be {A_B i , i=0,…,n0}, then for each target area in the target area set, the CNN feature extraction model is used to extract the feature vector, and the cosine similarity between the feature vectors is calculated using the following formula: Cos_Sim=(A_Feat·B_Feat) / (||A_Feat||*||B_Feat||), where A_Feat and B_Feat are A_Box ROI 、B_Box ROI If the similarity threshold Sim_Thr is set, when |Cos_Sim|>Sim_Thr, determine A_Box ROI Detection box and B_Box ROI The detection frames match, that is, the two detection frames may correspond to the same sorting object, thus obtaining a set of detection frames with similarity matching {A_B i , i=0,…,n1}, n1 is the number of matching pairs that satisfy both position matching and similarity matching. It can be understood that n1 is less than or equal to n0. Therefore, the number of target areas corresponding to the same sorting object can be determined based on the detection frames that match both similarity and distance. Still taking the above-mentioned first grid image and the second grid image corresponding to the position matching detection frame set as {A0_B0, A0_B1, A0_B2, A1_B0, A1_B1, A1_B2, A2_B0, A2_B2} as an example, the cosine similarity between each first target area and the second target area can be shown in the following Table 2:

[0114] Table 2

[0115]

[0116] If the similarity threshold Sim_Thr is 0.8, then the set of detection frames that match both the similarity and the position is {A0_B1, A1_B1, A1_B2, A2_B0}. Furthermore, the number of target areas corresponding to the same sorting object can be finally determined by sorting the similarities from large to small. In an embodiment of the present invention, one or more second target areas corresponding to each of the first target areas can be determined from the target area set; the one or more second target areas determined can be sorted according to the similarity; and the second target area corresponding to the same sorting object as the first target area can be determined according to the sorting result.

[0117] In this example, A_Box can be determined according to Table 2 ROI0 The corresponding second target area is B_Box ROI1 , A_Box ROI1 The corresponding second target area is B_Box ROI1 and B_Box ROI2 , A_Box ROI2 The corresponding second target area is B_Box ROI0 . Because A_Box ROI0 and A_Box ROI2 It only corresponds to one second target area, so there is no need to sort the second target area, and A_Box can be directly determined. ROI0 and B_Box ROI1 Corresponding to the same sorting object, A_Box ROI2 and B_Box ROI0 Corresponding to the same sorting object. And for A_Box ROI1 The corresponding second target area is B_Box ROI1 and B_Box ROI2 , sorted by similarity, we know that B_Box ROI2 Greater than B_Box ROI1 , so determine the B_Box with the greatest similarity ROI2 With A_Box ROI1 Corresponding to the same sorting object.

[0118] Therefore, after determining the first number (NA and NB) of target areas corresponding to the sorting areas and the boundary areas in the first grid image and the second grid image, and the second number (such as n2) corresponding to the same sorting object in the boundary area, the difference between the first number and the second number can be used as the current number of sorting objects corresponding to the sorting grid. That is, in the grid image spliced ​​by camera A and camera B, the current number of sorting objects corresponding to the grid image is NA+NB-n2. In this example, n2 is 3, and the current number of sorting objects corresponding to the grid image is NA+NB-3.

[0119] Step S104: allocating the sorting objects according to the current quantity of the sorting objects and the current sorting quantity corresponding to the sorting grid.

[0120] In an embodiment of the present invention, after determining the current sorting quantity of sorting objects in a sorting grid, the target quantity of sorting objects required by the sorting grid can be determined according to the ratio of the current sorting quantity to the standard workload; when the difference between the target quantity and the current quantity is greater than a preset first threshold, the sorting objects are transferred into the sorting grid; when the difference between the target quantity and the current quantity is less than a preset second threshold, the objects to be transferred out are determined from the sorting grid, and the transferred out objects are allocated to other sorting grids.

[0121] Among them, the first threshold and the second threshold can be the same or different, and their specific values ​​can be set according to actual needs. For example, the first threshold can be 3, and the second threshold can be -3. If the amount of sorted goods in the current batch of the sorting grid is M, the standard workload per capita of a single batch is set to M_thr, and the current number of sorting objects determined by the above embodiment is M2, then when M / M_thr-M2>3, the allocation method of sorting objects provided by the embodiment of the present invention can send a sorting object transfer notification to the grid process and display the transferred object ID. When M / M_thr-M2<-3, the allocation method of sorting objects provided by the embodiment of the present invention randomly selects a sorting object to be transferred to the grid process and sends the ID information of the sorting grid process to which it is transferred. Among them, the sorting object can be a sorting personnel or a sorting equipment.

[0122] After the sorting objects are allocated, the sorting allocation method provided by the embodiment of the present invention can also automatically record the workload and working hours of each sorting object. Furthermore, when the sorting object is a sorting personnel, the embodiment of the present invention can also use a classification model to identify the type of sorting object, for example, according to the work clothes worn by the sorting personnel, the sorting personnel are divided into self-employed employees (standard work clothes) and outsourced personnel (other clothes except standard work clothes), and the number of different types of personnel identified is submitted to the online system of the sorting site (a system that can provide the sorting object allocation method of the embodiment of the present invention), thereby facilitating the online system to allocate the sorting objects according to the current sorting volume and the average workload per type of work in the sorting grid, realizing automated manpower coordination according to personnel categories, and automatically recording the attendance data, workload, working hours and other data of sorting personnel of different categories.

[0123] The following takes the collection screen of a camera corresponding to a sorting grid and the collection screen of multiple cameras corresponding to a sorting grid as examples to describe in detail the sorting object allocation method provided by the embodiment of the present invention. Figure 3 As shown, when the sorting grid corresponds to a collection picture of a camera, the method for allocating sorting objects provided by the embodiment of the present invention includes the following steps:

[0124] Step S301: Acquire a video stream captured by the camera device, and extract frames from the video stream according to a preset frame extraction interval to obtain the grid image.

[0125] Step S302: marking a sorting area corresponding to a sorting grid in the grid image, and obtaining an image set corresponding to the sorting grid according to the marking result.

[0126] Step S303: input the grid image into a detection model based on a convolutional neural network, and determine one or more object regions, location information and confidence of the object regions according to the output of the detection model.

[0127] Among them, the detection model is trained based on logistics images annotated with logistics objects and object labels.

[0128] Step S304: According to the position information and confidence of the object area, determine, from one or more object areas, an object area of ​​the target whose confidence is greater than a preset confidence threshold and is located in the sorting area corresponding to the sorting grid.

[0129] Step S305: determining the mode of the target areas according to the number of target areas in each of the grid images in the image set, and determining the current number of sorting objects corresponding to the sorting grid according to the mode.

[0130] Step S306: Determine the target number of sorting objects required for the sorting grid according to the ratio of the current sorting volume corresponding to the sorting grid to the standard workload; if the difference between the target number and the current number is greater than a preset first threshold, execute step S307; if the difference between the target number and the current number is less than a preset second threshold, execute step S308; if the difference is not greater than the first threshold and not less than the second threshold, end the current process.

[0131] exist Figure 3 The difference between the target quantity and the current quantity is represented by M, the first threshold is represented by K1, and the second threshold is represented by K2.

[0132] Step S307: calling sorting objects into the sorting grid;

[0133] Here, the sorting objects may be transferred from other sorting grids where there are redundant sorting objects. It is understandable that there are multiple sorting grids in the sorting site, and the sorting grids where there are redundant sorting objects are the sorting grids where the above difference is less than the second threshold.

[0134] Step S308: determining a call-out object from the sorting grid, and allocating the call-out object to other sorting grids.

[0135] like Figure 4 As shown, when the sorting grid corresponds to the collection images of two cameras, the method for allocating sorting objects provided by the embodiment of the present invention includes the following steps:

[0136] Step S401: Determine two camera devices corresponding to the sorting grid, and obtain video streams respectively collected by the two camera devices; extract frames from the video streams collected by the two camera devices respectively according to the same frame extraction interval to obtain a grid image.

[0137] Step S402: for a group of grid images corresponding to the same frame extraction interval: marking the sorting area and the boundary areas corresponding to the multiple camera devices in the grid images, and obtaining an image set corresponding to the sorting grid according to the marking results.

[0138] Step S403: input the grid image into a detection model based on a convolutional neural network, and determine one or more object regions, location information and confidence of the object regions according to the output of the detection model.

[0139] Step S404: For multiple frames of grid images corresponding to any camera: determine the mode of the target areas and the mode frame corresponding to the mode according to the number of target areas in each of the grid images, and determine the target frame with the largest confidence mean from the mode frames according to the confidence mean corresponding to the target areas of the mode frames.

[0140] Step S405: Determine a second grid image having a boundary area with the target frame from a group of grid images corresponding to the same frame extraction interval, determine a first target area in the boundary area of ​​the best frame, and determine a second target area in the boundary area of ​​the second grid image.

[0141] Step S406: Determine the relative distance between the first target area and the second target area.

[0142] Step S407: For the first target area and the second target area whose relative distance is less than a preset distance threshold: construct a target area set based on the first target area and the second object area whose relative distance to the first target area is less than the preset distance threshold; use a feature extraction model to extract feature vectors of each target object area in the target area set, and calculate the cosine similarity between the first target area and the second target area in the target area set based on the feature vectors.

[0143] Step S408: determining one or more second target areas corresponding to each of the first target areas from the target area set, and determining a second target area corresponding to the same sorting object as the first target area according to the sorting result.

[0144] Step S409: determining a first number of target areas corresponding to the sorting areas and the boundary areas in the first grid image and the second grid image, and a second number of target areas corresponding to the same sorting object in the boundary area;

[0145] Step S410: taking the difference between the first quantity and the second quantity as the current quantity of the sorting objects corresponding to the sorting grid.

[0146] Step S411: Determine the target number of sorting objects required for the sorting grid according to the ratio of the current sorting volume corresponding to the sorting grid to the standard workload; if the difference between the target number and the current number is greater than a preset first threshold, execute step S412; if the difference between the target number and the current number is less than a preset second threshold, execute step S413; if the difference is not greater than the first threshold and not less than the second threshold, end the current process.

[0147] exist Figure 4 The difference between the target quantity and the current quantity is represented by M, the first threshold is represented by K1, and the second threshold is represented by K2.

[0148] Step S412: calling sorting objects into the sorting grid;

[0149] Here, the sorting objects may be transferred from other sorting grids where there are redundant sorting objects. It is understandable that there are multiple sorting grids in the sorting site, and the sorting grids where there are redundant sorting objects are the sorting grids where the above difference is less than the second threshold.

[0150] Step S413: determining a call-out object from the sorting grid, and allocating the call-out object to other sorting grids.

[0151] According to a method for allocating sorting objects according to an embodiment of the present invention, it can be seen that the target area corresponding to the sorting object is identified from the image set corresponding to the acquired sorting grid, and then the current number of sorting objects corresponding to the sorting grid is determined according to the number and / or position of the target area, and then the sorting objects are allocated according to the current number of sorting objects in the sorting grid and the current sorting quantity. In this way, the sorting objects in the sorting grid are automatically and accurately identified, and then the sorting objects are accurately and reasonably allocated according to the identification results, so that the allocation of the sorting objects can adapt to the changes in the sorting quantity, which is conducive to balancing the workload of each sorting object and improving the sorting efficiency.

[0152] Figure 5 4 is a schematic diagram of main modules of a device for allocating sorting objects according to an embodiment of the present invention.

[0153] like Figure 5 As shown, a sorting object allocation device 500 according to an embodiment of the present invention includes: an image acquisition module 501, an area determination module 502, a quantity determination module 503 and an allocation module 504; wherein,

[0154] The image acquisition module 501 is used to acquire an image set corresponding to a sorting grid, wherein the image set includes one or more grid images corresponding to the sorting grid;

[0155] The region determination module 502 is used to respectively identify target regions corresponding to the sorting objects from one or more grid images;

[0156] The quantity determination module 503 is used to determine the current quantity of sorting objects corresponding to the sorting grid according to at least one of the quantity of target areas in each of the grid images and the position of the target areas in the grid images;

[0157] The allocation module 504 is used to allocate the sorting objects according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid.

[0158] In one embodiment of the present invention, the region determination module 502 is used to input the grid image into a detection model based on a convolutional neural network, where the detection model is trained based on logistics images marked with logistics objects and object labels; determine one or more object regions, location information and confidence of the object regions according to the output of the detection model; and determine the target region corresponding to the sorting object in the grid image according to at least one of the location information and confidence of each of the object regions.

[0159] In one embodiment of the present invention, the area determination module 502 is used to: determine, for each of the grid images, the center point of the object area and the angular coordinates of the object area according to the position information; determine whether the object area is located in the sorting area corresponding to the sorting grid according to at least one of the following: whether the center point of the target area is located in the sorting area, and whether the number of target angular coordinates of the target area is not less than a preset number threshold; wherein the target angular coordinates are angular coordinates located in the sorting area; if it is determined that the object area is located in the sorting area, determine the object area as the target area corresponding to the sorting object, and increment the number of target areas in the grid image.

[0160] In one embodiment of the present invention, the quantity determination module 503 is used to determine the mode of the target areas according to the number of target areas in each of the grid images in the image set; and determine the current number of sorting objects corresponding to the sorting grid according to the mode.

[0161] In one embodiment of the present invention, the quantity determination module 503 is used to determine the grid image corresponding to each mode when there are multiple modes, and calculate the confidence mean of the grid image; use the grid image with the largest confidence mean as the target grid image in the image set, and determine the current quantity of the sorting objects according to the object area included in the target grid image;

[0162] In one embodiment of the present invention, the quantity determination module 503 is configured to use the largest majority as the current quantity of the sorting object when there are multiple majority values.

[0163] In one embodiment of the present invention, the region determination module 502 is used to determine the target region corresponding to the sorting object according to the object region whose confidence is greater than a preset confidence threshold.

[0164] In one embodiment of the present invention, the image acquisition module 501 is used to determine the camera device corresponding to the sorting grid and obtain the video stream captured by the camera device; extract frames from the video stream according to a preset frame extraction interval to obtain the grid image; mark the sorting area corresponding to the sorting grid in the grid image, and obtain the image set corresponding to the sorting grid according to the marking result.

[0165] In one embodiment of the present invention, the image acquisition module 501 is used to, when there are multiple cameras corresponding to the sorting grid, respectively extract frames for the video streams collected by the multiple cameras at the same frame extraction interval to obtain the grid image; for a group of grid images corresponding to the same frame extraction interval: mark the sorting area and the boundary areas corresponding to the multiple cameras in the grid image; and obtain the image set corresponding to the sorting grid according to the marking result.

[0166] In one embodiment of the present invention, the quantity determination module 503 is used to determine a first grid image and a second grid image intersecting with the first grid image from each group of the grid images; determine a first target area in the intersecting area of ​​the first grid image according to the position of the target area in the grid images, and determine a second target area in the intersecting area of ​​the second grid image; determine a relative distance between the first target area and the second target area; and determine whether the first target area and the second target area correspond to the same sorting object according to the relative distance.

[0167] In one embodiment of the present invention, the quantity determination module 503 is used to determine a first position coefficient of the first target area relative to the boundary area of ​​the first grid image, and a second position coefficient of the second target area relative to the boundary area of ​​the second grid image; and determine the relative distance based on the difference between the first position coefficient and the second position coefficient.

[0168] In one embodiment of the present invention, the quantity determination module 503 is used to: calculate the similarity between the first target area and the second target area whose relative distance is less than a preset distance threshold; and determine whether the first target area and the second target area correspond to the same sorting object according to the similarity.

[0169] In one embodiment of the present invention, the quantity determination module 503 is used to construct a target area set based on the first target area and the second object area whose relative distance to the first target area is less than a preset distance threshold; use a feature extraction model to extract feature vectors of each target object area in the target area set; and calculate the cosine similarity between the first target area and the second target area in the target area set based on the feature vector.

[0170] In one embodiment of the present invention, the quantity determination module 503 is used to determine, from the target area set, one or more second target areas corresponding to each of the first target areas when there are multiple first target areas and multiple second target areas respectively; sort the determined one or more second target areas according to the similarity; and determine, according to the sorting result, a second target area corresponding to the same sorting object as the first target area.

[0171] In one embodiment of the present invention, the quantity determination module 503 is used to determine, for a multi-frame grid image corresponding to any camera: the mode of the target area in the multi-frame grid image, and the mode frame corresponding to the mode; determine the target frame with the largest confidence mean from the mode frames according to the confidence mean corresponding to the target area of ​​the mode frame; and use the target frame as the first grid image.

[0172] In one embodiment of the present invention, the quantity determination module 503 is used to determine a first quantity of target areas corresponding to the sorting areas and boundary areas in the first grid image and the second grid image; determine a second quantity corresponding to the same sorting object in the boundary area; and use the difference between the first quantity and the second quantity as the current quantity of the sorting objects corresponding to the sorting grid.

[0173] In one embodiment of the present invention, the allocation module 504 is used to determine the target number of sorting objects required by the sorting grid according to the ratio of the current sorting volume to the standard workload; when the difference between the target number and the current number is greater than a preset first threshold, call in the sorting objects for the sorting grid; when the difference between the target number and the current number is less than a preset second threshold, determine the objects to be called out from the sorting grid and allocate the called out objects to other sorting grids.

[0174] In one embodiment of the present invention, the allocation module 504 is further used to: record the workload and working time of each of the sorting objects.

[0175] It can be seen from a device for allocating sorting objects according to an embodiment of the present invention that it is possible to identify the target area corresponding to the sorting object from the image set corresponding to the acquired sorting grid, and then determine the current number of sorting objects corresponding to the sorting grid according to the number and / or position of the target area, and then allocate the sorting objects according to the current number of sorting objects in the sorting grid and the current sorting quantity. This realizes automatic and accurate identification of sorting objects in the sorting grid, and then accurately and reasonably allocates the sorting objects according to the identification results, so that the allocation of sorting objects can adapt to the changes in the sorting quantity, which is conducive to balancing the workload of each sorting object and improving the sorting efficiency.

[0176] Figure 6 An exemplary system architecture 600 of a method for allocating sorted objects or a device for allocating sorted objects according to an embodiment of the present invention is shown.

[0177] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0178] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as sorting task receiving and query applications, route navigation applications, sorting grid planning applications, workload recording applications, etc.

[0179] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0180] The server 605 may be a server that provides various services, such as a background management server that provides support for the workload queried by the user using the terminal devices 601, 602, and 603. The background management server may analyze and process the received query request and other data, and feed back the processing results (such as the workload corresponding to each sorting grid) to the terminal device.

[0181] It should be noted that the sorting object allocation method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the sorting object allocation device is generally arranged in the server 605 .

[0182] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0183] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0184] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0185] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0186] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0187] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0188] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0189] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an image acquisition module, a region determination module, a quantity determination module and a deployment module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the deployment module may also be described as a "module for deploying sorting objects".

[0190] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining an image set corresponding to a sorting grid, the image set including one or more grid images corresponding to the sorting grid; identifying target areas corresponding to sorting objects from one or more grid images; determining the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each grid image and the position of the target area in the grid image; and allocating the sorting objects according to the current number of sorting objects and the current sorting amount corresponding to the sorting grid.

[0191] According to the technical solution of the embodiment of the present invention, the target area corresponding to the sorting object can be identified from the image set corresponding to the acquired sorting grid, and then the current number of sorting objects corresponding to the sorting grid can be determined according to the number and / or position of the target area, and then the sorting objects can be allocated according to the current number of sorting objects in the sorting grid and the current sorting quantity. In this way, the sorting objects in the sorting grid can be automatically and accurately identified, and then the sorting objects can be accurately and reasonably allocated according to the identification results, so that the allocation of the sorting objects can adapt to the changes in the sorting quantity, which is conducive to balancing the workload of each sorting object and improving the sorting efficiency.

[0192] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for allocating sorting objects, characterized in that: include: Acquire an image set corresponding to a sorting grid, wherein the image set includes one or more grid images corresponding to the sorting grid; Identifying target areas corresponding to the sorting objects from one or more of the grid images respectively; Determine the current number of sorting objects corresponding to the sorting grid according to at least one of the number of target areas in each of the grid images and the position of the target area in the grid image; The sorting objects are allocated according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid.

2. The method according to claim 1, characterized in that The step of respectively identifying target areas corresponding to the sorting objects from one or more grid images comprises: Inputting the grid image into a detection model based on a convolutional neural network, wherein the detection model is trained based on logistics images annotated with logistics objects and object labels; Determine one or more object regions, location information and confidence of the object regions according to the output of the detection model; According to at least one of the position information and the confidence level of each of the object regions, a target region corresponding to the sorting object in the grid image is determined.

3. The method according to claim 2, characterized in that Determining the target area corresponding to the sorting object in the grid image according to at least one of the position information and the confidence of each of the object areas includes: For each of the grid images: determining the center point of the object area and the angular coordinates of the object area according to the position information; Determine whether the object area is located in the sorting area corresponding to the sorting grid according to at least one of the following: whether the center point of the target area is located in the sorting area, and whether the number of target angular coordinates of the target area is not less than a preset number threshold; wherein the target angular coordinate is an angular coordinate located in the sorting area; In the case where it is determined that the object area is located within the sorting area, the object area is determined to be a target area corresponding to the sorting object, and the number of target areas in the grid image is incremented.

4. The method according to claim 3, characterized in that: The determining, according to at least one of the number of target areas in each of the grid images and the position of the target area in the grid image, the current number of sorting objects corresponding to the sorting grid comprises: Determining the mode of the target areas according to the number of target areas in each of the grid images in the image set; The current number of sorting objects corresponding to the sorting grid is determined according to the majority.

5. The method according to claim 4, characterized in that In the case where there are multiple modes, determining the current number of sorting objects corresponding to the sorting grid according to the modes includes: Determine the grid image corresponding to each mode, and calculate the confidence mean of the grid image; use the grid image with the largest confidence mean as the target grid image in the image set, and determine the current number of the sorted objects according to the object area included in the target grid image; or, The maximum number is taken as the current number of the sorting objects.

6. The method according to claim 1, characterized in that The step of obtaining an image set corresponding to a sorting grid comprises: Determine the camera device corresponding to the sorting grid, and obtain the video stream captured by the camera device; Extract frames from the video stream according to a preset frame extraction interval to obtain the grid image; The sorting area corresponding to the sorting grid is marked in the grid image, and the image set corresponding to the sorting grid is obtained according to the marking result.

7. The method according to claim 6, characterized in that In the case where the sorting grid corresponds to a plurality of camera devices, the step of obtaining the image set corresponding to the sorting grid includes: Extract frames from the video streams collected by the multiple camera devices respectively according to the same frame extraction interval to obtain the grid image; For a group of grid images corresponding to the same frame extraction interval: marking the boundary areas corresponding to the sorting area and the plurality of camera devices in the grid images; An image set corresponding to the sorting grid is obtained according to the labeling result.

8. The method according to claim 7, characterized in that The determining, according to at least one of the number of target areas in each of the grid images and the position of the target area in the grid image, the current number of sorting objects corresponding to the sorting grid comprises: Determine a first grid image and a second grid image that intersects with the first grid image from each group of the grid images; Determine a first target area in a boundary area of ​​the first grid image according to a position of the target area in the grid image, and determine a second target area in a boundary area of ​​the second grid image; determining a relative distance between the first target area and the second target area; According to the relative distance, it is determined whether the first target area and the second target area correspond to the same sorting object.

9. The method according to claim 8, characterized in that Include at least one of the following: The determining the relative distance between the first target area and the second target area comprises: determining a first position coefficient of the first target area relative to a boundary area of ​​the first grid image, and a second position coefficient of the second target area relative to a boundary area of ​​the second grid image; and determining the relative distance according to a difference between the first position coefficient and the second position coefficient; The determining, based on the relative distance, whether the first target area and the second target area correspond to the same sorting object comprises: for the first target area and the second target area whose relative distance is less than a preset distance threshold: calculating the similarity between the first target area and the second target area; and determining, based on the similarity, whether the first target area and the second target area correspond to the same sorting object.

10. The method according to claim 9, characterized in that The calculating the similarity between the first target area and the second target area includes: Constructing a target area set according to the first target area and a second object area whose relative distance to the first target area is less than a preset distance threshold; Extracting feature vectors of each target object region in the target region set using a feature extraction model; The cosine similarity between the first target region and the second target region in the target region set is calculated according to the feature vector.

11. The method according to claim 8, characterized in that Include at least one of the following: For a multi-frame grid image corresponding to any camera: determining the mode of the target area in the multi-frame grid image and the mode frame corresponding to the mode; determining the target frame with the largest confidence mean from the mode frames according to the confidence mean corresponding to the target area of ​​the mode frame; Using the target frame as the first grid image; The determining the current number of sorting objects corresponding to the sorting grid includes: determining a first number of target areas corresponding to the sorting areas and boundary areas in the first grid image and the second grid image; Determine a second quantity corresponding to the same sorting object in the boundary area; and use the difference between the first quantity and the second quantity as the current quantity of the sorting object corresponding to the sorting grid.

12. The method according to claim 1, characterized in that Include at least one of the following: The allocating of the sorting objects according to the number of the sorting objects and the current sorting volume corresponding to the sorting grid includes: determining the target number of sorting objects required by the sorting grid according to the ratio of the current sorting volume to the standard workload; calling in sorting objects to the sorting grid when the difference between the target number and the current number is greater than a preset first threshold; and determining objects to be called out from the sorting grid when the difference between the target number and the current number is less than a preset second threshold, and allocating the called out objects to other sorting grids; After allocating the sorting objects according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid, the method further includes: recording the workload and working time of each of the sorting objects.

13. A device for dispensing sorted objects, characterized in that: include: Image acquisition module, area determination module, quantity determination module and deployment module; wherein, The image acquisition module is used to acquire an image set corresponding to a sorting grid, wherein the image set includes one or more grid images corresponding to the sorting grid; The area determination module is used to respectively identify target areas corresponding to the sorting objects from one or more of the grid images; The quantity determination module is used to determine the current quantity of sorting objects corresponding to the sorting grid according to at least one of the quantity of target areas in each of the grid images and the position of the target areas in the grid images; The allocation module is used to allocate the sorting objects according to the current number of the sorting objects and the current sorting quantity corresponding to the sorting grid.

14. An electronic device for sorting objects, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

15. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images