Accumulation degree determination method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a stacking degree determination method and device, equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: inputting a region image of a to-be-recognized region into a pre-trained division model, the region image is divided into a plurality of pixel regions by the division model, the region type of each pixel region is determined, the division model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region; sorting the pixel regions of which the region types are non-interference regions based on the region areas, and determining an area threshold value according to a sorting result; and according to the area threshold, determining an article area in each pixel area of which the area type is a non-interference area, and according to the total area of the article area and the area of the to-be-identified area, determining the article stacking degree of the to-be-identified area. According to the technical scheme, the article stacking degree of the to-be-identified area is accurately determined.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a method, device, equipment and storage medium for determining a stacking degree. Background Art

[0002] In logistics scenarios, the degree of accumulation of items in the storage area can reflect the degree of congestion in the storage area. When the degree of accumulation of items in the storage area is high, it will cause congestion, resulting in low logistics efficiency. In addition, the accumulation of messy items is also prone to cause loss and damage of items, posing a threat to the safety of people and items. Therefore, timely grasping the degree of accumulation of items in the storage area is an important task to ensure logistics efficiency.

[0003] In the prior art, an object area is usually determined in a regional image corresponding to a storage area based on a target detection model, and the degree of object accumulation is determined according to the proportion of the object area in the regional image.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art:

[0005] The object detection model has a high training cost and low generalization, resulting in low accuracy in determining the degree of object accumulation. Summary of the invention

[0006] The present invention provides a method, device, equipment and storage medium for determining the degree of accumulation of objects in a to-be-identified area, so as to achieve accurate determination of the degree of accumulation of objects in a to-be-identified area.

[0007] In a first aspect, an embodiment of the present invention provides a method for determining a stacking degree, comprising:

[0008] Inputting a region image of the region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region;

[0009] Sorting the pixel regions whose region types are non-interference regions based on the region areas, and determining the area threshold according to the sorting result;

[0010] An object area is determined in each of the pixel areas whose area type is a non-interference area according to the area threshold, and an object accumulation degree of the area to be identified is determined according to a total area of ​​the object area and an area of ​​the area to be identified.

[0011] In a second aspect, an embodiment of the present invention further provides a device for determining a stacking degree, comprising:

[0012] An input module, used for inputting a region image of a region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region;

[0013] A sorting module, used for sorting the pixel regions whose region type is a non-interference region based on the region area, and determining the area threshold according to the sorting result;

[0014] The determination module is used to determine the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold, and determine the degree of object accumulation in the area to be identified according to the total area of ​​the object area and the area of ​​the area to be identified.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the degree of accumulation as described in any one of the first aspects.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute a method for determining a degree of accumulation as described in any one of the first aspects.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] An embodiment of the present invention provides a method for determining a degree of accumulation, comprising: inputting a regional image of a to-be-identified area into a pre-trained segmentation model, so that the segmentation model divides the regional image into a plurality of pixel areas and determines the area type of each of the pixel areas, wherein the segmentation model is composed of a classification module and a segmentation model, and the area type is an interference area or a non-interference area; sorting each of the pixel areas whose area type is a non-interference area based on the area area, and determining an area threshold according to the sorting result; determining an object area in each of the pixel areas whose area type is a non-interference area according to the area threshold, and determining the degree of object accumulation in the to-be-identified area according to the total area of ​​the object area and the area of ​​the to-be-identified area. According to the above technical solution, after obtaining the regional image of the area to be identified, the regional image is input into a pre-trained segmentation model. The segmentation model can divide the regional image into multiple pixel areas according to the objects contained in the regional image, and can also determine whether the area type of each pixel area is an interference area or a non-interference area, so as to achieve the goal of dividing the regional image into multiple pixel areas and determining the area type of each pixel area at a relatively low cost, determining the area area of ​​each pixel area whose area type is a non-interference area, sorting each pixel area whose area type is a non-interference area based on the area area, determining the area threshold according to the sorting result, and achieving the goal of determining the area threshold for determining whether the pixel area whose area type is a non-interference area is an object area according to the sorting result, providing data reference for determining the object area in each pixel area whose area type is a non-interference area, and determining the object area in each pixel area whose area type is a non-interference area according to the area threshold. Compared with the prior art of determining the object area according to the target detection technology, the present application has a lower cost for determining the object area, and has a higher accuracy and a stronger versatility. After determining the object area, the object accumulation degree of the area to be identified can be determined according to the total area of ​​the object area and the area of ​​the area to be identified, so as to achieve the accurate determination of the object accumulation degree of the area to be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a method for determining a stacking degree provided by an embodiment of the present invention;

[0023] Figure 2 A flow chart of another method for determining a stacking degree provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a partitioning model in another method for determining a stacking degree provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of an area curve constructed according to a sorting result in another method for determining a stacking degree provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the structure of a device for determining a stacking degree provided by an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0030] In the prior art, methods for determining the degree of accumulation of items in a storage area can be divided into two categories. One category can determine the degree of accumulation of items in a storage area manually, and the other category can determine the degree of accumulation of items in a storage area through computer vision. Specifically, a regional image of the storage area can be obtained, and the degree of accumulation of items in the storage area can be determined based on the regional image.

[0031] The problems of manually determining the degree of accumulation of items in a storage area are: 1. Frequent inspections of the storage area are required, which consumes additional manpower; 2. When the storage area is large, it is difficult for humans to form a global view and determine the accurate degree of accumulation of items. Compared with manual determination, computer vision can effectively save manpower when determining the degree of accumulation of items in a storage area, and can form a global view of the storage area when the regional image of the storage area is obtained. At present, computer vision mainly determines the object area in the regional image corresponding to the storage area based on target detection technology, and determines the degree of accumulation of items based on the proportion of the object area in the regional image. However, this method requires the collection of field data in advance to train the target detection model, and the cost of data collection, labeling and model training is high. In addition, due to the obvious differences between storage areas and between items, when deploying the model in a new storage area, it is often necessary to re-collect field data to tune the model.

[0032] Therefore, the present application proposes a method for determining the degree of accumulation of objects in the area to be identified, so as to accurately determine the degree of accumulation of objects in the area to be identified.

[0033] The method for determining the stacking degree proposed in the present application will be described in detail below with reference to diagrams and embodiments.

[0034] Figure 1 The flowchart of a method for determining the degree of accumulation provided by an embodiment of the present invention is applicable to the situation where the degree of accumulation of objects in the area to be identified needs to be accurately determined. The method can be executed by a device for determining the degree of accumulation, which can be implemented by software and / or hardware. Figure 1 The method specifically comprises the following steps:

[0035] Step 110: input the region image of the area to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region.

[0036] The partition model is composed of a classification module and a segmentation model, the region type is an interference region or a non-interference region, and the segmentation model may be a Segment Anything Model (SAM).

[0037] The segmentation model composed of the classification module and the segmentation model is used to segment the image into multiple pixel areas according to the items contained in the image and determine the area type of each pixel area. The area type can be an interference area or a non-interference area. When the area to be identified is a storage area in a logistics scenario, the interference area can be understood as an area containing non-static items, for example, an area containing vehicles or containing humans, and the non-interference area can be understood as an area containing static items or no items, for example, an area containing a conveyor belt or a shelf, or an empty area. Since the segmentation model already has a strong image recognition capability, the image is input into the segmentation model, and the segmentation model can segment the image into multiple pixel areas according to the items contained in the image. Therefore, after constructing the initial segmentation model according to the classification module and the segmentation model, only the parameters of the classification module can be adjusted, and the segmentation model can be reused for the rest. In addition, the classification module can be constructed with a shallow neural network, so that the training and use of the segmentation model will not increase the computational cost too much.

[0038] Specifically, after obtaining the regional image of the area to be identified, the regional image can be input into a pre-trained segmentation model. The segmentation model can divide the regional image into multiple pixel regions according to the objects contained in the regional image, and can also determine the region type of each pixel region as an interference region or a non-interference region.

[0039] In an embodiment of the present invention, after the regional image of the area to be identified is input into the segmentation model, the segmentation model can divide the regional image into multiple pixel areas and determine the area type of each pixel area, thereby dividing the regional image into multiple pixel areas and determining the area type of each pixel area at a lower cost.

[0040] Step 120: sort the pixel regions whose region type is a non-interference region based on the region area, and determine the area threshold according to the sorting result.

[0041] Non-static items in the interference area can leave the area to be identified at any time. Therefore, when determining the accumulation level of items in the area to be identified, the interference area needs to be excluded. Since the non-interference area also includes the area containing static equipment and the area not containing any items, and the area containing static equipment and the area not containing any items are significantly larger than the area containing items, the area area of ​​each pixel area whose area type is the non-interference area can be determined, and the item area can be determined in each pixel area whose area type is the non-interference area according to the area area.

[0042] Specifically, after determining the region type of each pixel region in the region image, the region area of ​​each pixel region whose region type is a non-interference region can be determined, and the pixel regions whose region type is a non-interference region can be sorted based on the region area, and then the area threshold can be determined according to the sorting result. This area threshold can be used to determine whether the pixel region whose region type is a non-interference region is an object region.

[0043] In practical applications, after the pixel regions whose region types are non-interference regions are sorted based on the region areas, the smaller region area among the adjacent and sharply decreasing region areas may be determined as the area threshold.

[0044] In an embodiment of the present invention, after determining the area of ​​each pixel area whose area type is a non-interference area, the pixel areas whose area type is a non-interference area are sorted based on the area area, and an area threshold is determined according to the sorting result. The area threshold can be used to determine whether the pixel area whose area type is a non-interference area is an object area, and provide data reference for determining the object area in each pixel area whose area type is a non-interference area.

[0045] Step 130: determining an object area in each of the pixel areas whose area type is a non-interference area according to the area threshold, and determining the degree of object accumulation in the area to be identified according to the total area of ​​the object area and the area of ​​the area to be identified.

[0046] Specifically, the pixel areas whose area area is less than the area threshold in each pixel area whose area type is a non-interference area can be determined as the object area. Furthermore, the total area of ​​the object area can be determined according to the area area of ​​each object area, and the degree of object accumulation in the area to be identified can be determined according to the ratio of the total area of ​​the object area to the area of ​​the area to be identified.

[0047] In the embodiment of the present invention, the object area is determined in each pixel area whose area type is a non-interference area according to the area threshold, and the degree of object accumulation in the area to be identified is further determined according to the total area of ​​the object area and the area of ​​the area to be identified.

[0048] The method for determining the degree of accumulation provided by an embodiment of the present invention includes: inputting a regional image of a to-be-identified area into a pre-trained segmentation model so that the segmentation model divides the regional image into a plurality of pixel areas and determines the area type of each of the pixel areas, wherein the segmentation model is composed of a classification module and a segmentation model, and the area type is an interference area or a non-interference area; sorting the pixel areas whose area type is a non-interference area based on the area area, and determining an area threshold according to the sorting result; determining an object area in the pixel areas whose area type is a non-interference area according to the area threshold, and determining the degree of object accumulation in the to-be-identified area according to the total area of ​​the object area and the area of ​​the to-be-identified area. According to the above technical solution, after obtaining the regional image of the area to be identified, the regional image is input into a pre-trained segmentation model. The segmentation model can divide the regional image into multiple pixel areas according to the objects contained in the regional image, and can also determine whether the area type of each pixel area is an interference area or a non-interference area, so as to achieve the goal of dividing the regional image into multiple pixel areas and determining the area type of each pixel area at a relatively low cost, determining the area area of ​​each pixel area whose area type is a non-interference area, sorting each pixel area whose area type is a non-interference area based on the area area, determining the area threshold according to the sorting result, and achieving the goal of determining the area threshold for determining whether the pixel area whose area type is a non-interference area is an object area according to the sorting result, providing data reference for determining the object area in each pixel area whose area type is a non-interference area, and determining the object area in each pixel area whose area type is a non-interference area according to the area threshold. Compared with the prior art of determining the object area according to the target detection technology, the present application has a lower cost for determining the object area, and has a higher accuracy and a stronger versatility. After determining the object area, the object accumulation degree of the area to be identified can be determined according to the total area of ​​the object area and the area of ​​the area to be identified, so as to achieve the accurate determination of the object accumulation degree of the area to be identified.

[0049] Figure 2A flowchart of another method for determining the degree of accumulation provided by an embodiment of the present invention. The embodiment of the present invention can be applied to situations where the degree of accumulation of items in the to-be-identified area needs to be accurately determined. Based on the above embodiment, after determining the degree of accumulation of items in the to-be-identified area according to the total area of ​​the item area and the area of ​​the to-be-identified area, the embodiment of the present invention adds "when determining that the degree of accumulation of items is greater than the degree threshold, determining that the items in the to-be-identified area are excessively accumulated." and also adds a training process for the partitioning model. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. See Figure 2 , the method for determining the degree of accumulation provided by the embodiment of the present invention includes:

[0050] Step 210: input the region image of the area to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region.

[0051] The division model is composed of a classification module and a segmentation model, and the area type is an interference area or a non-interference area.

[0052] There may be non-static objects entering and exiting the area to be identified, for example, vehicles and humans. Non-static objects such as vehicles and humans are not counted as accumulated objects. If no distinction is made, the area containing vehicles or humans will be misjudged as an object area, affecting the accuracy of the determined object accumulation degree. Therefore, it is necessary to distinguish the area containing non-static objects.

[0053] Since the segmentation model itself does not have the ability to segment instances, it is impossible to determine whether each area contains non-static items, so other modules need to be added. Adding additional non-static item recognition modules (for example, target detection or instance segmentation modules) will introduce additional computational complexity, and the recognition area of ​​the new module is also difficult to be completely consistent with the segmentation area of ​​the segmentation model. Therefore, the present application improves the segmentation model through a classification module, so that the improved partition model has the ability to recognize non-static items, and gives full play to the advantages of the generalization of the segmentation model, only adds a small number of parameters, and the additional training cost is low.

[0054] Figure 3 A schematic diagram of a partitioning model in another method for determining a stacking degree provided by an embodiment of the present invention, such as Figure 3 As shown, the partition model is composed of a classification module and a segmentation model, and the input of the classification module is the output of the fully connected layer in the segmentation model, that is, the region vector of each pixel region of the image input to the segmentation model, and the output of the classification module is the probability that each pixel region of the image input to the segmentation model is a region containing non-static objects. When the probability is determined to be greater than a preset threshold, the pixel region is determined to be an interference region containing non-static objects.

[0055] Specifically, after obtaining the regional image of the area to be identified, the regional image can be input into a pre-trained segmentation model. The segmentation model can divide the regional image into multiple pixel regions according to the objects contained in the regional image, and can also determine the region type of each pixel region as an interference region or a non-interference region.

[0056] When the acquired image contains the area to be identified and other areas, the image and the prompt word can be input into the segmentation model together. The prompt word can be a rectangular frame or coordinate information used to frame the area to be identified. When the prompt word is a rectangular frame, the segmentation model can divide the part of the area image that overlaps with the rectangular frame into multiple pixel areas according to the rectangular frame contained in the prompt word, and determine the area type of each pixel area as an interference area or a non-interference area; when the prompt word is coordinate information, the segmentation model can divide the part of the area image framed by the coordinate information into multiple pixel areas according to the coordinate information contained in the prompt word, and determine the area type of each pixel area as an interference area or a non-interference area.

[0057] In one implementation, the training process of the partitioning model includes:

[0058] Acquire a training image, process the training image based on the segmentation model, and obtain the region vector of each segmentation area corresponding to the training image; determine the region type of each segmentation area corresponding to each training image; use the region vector of each segmentation area corresponding to each training image as the training input of the classification module in the initial segmentation model, and use the region type of each segmentation area to guide the training output to perform network training on the classification module to obtain the segmentation model, wherein the initial segmentation model is composed of the classification module and the segmentation model.

[0059] The initial partitioning model does not change the internal structure and parameters of the segmentation model, and only adds a classification module. Therefore, the training of the initial partitioning model only adjusts the parameters of the classification module, and the rest of the segmentation model is reused.

[0060] Specifically, a training image is obtained, which may be a regional image containing static objects and / or non-static objects. The training image is processed based on the segmentation model, and the fully connected layer can output the regional vectors of each segmented region corresponding to the training image. The regional type of each segmented region corresponding to each training image is determined. When the regional type of the segmented region is an interference region, its label is determined to be 1, and when the regional type of the segmented region is a non-interference region, its label is determined to be 0. The regional vectors of each segmented region corresponding to each training image are used as the training input of the classification module in the initial segmentation model, and the regional type of each segmented region is used to guide the training output to perform network training on the classification module, so as to obtain a segmentation model.

[0061] When training the network for the classification module, cross entropy loss and stochastic gradient descent can be used for optimization.

[0062] In an embodiment of the present invention, after the regional image of the area to be identified is input into the segmentation model, the segmentation model can divide the regional image into multiple pixel areas and determine the area type of each pixel area, thereby dividing the regional image into multiple pixel areas and determining the area type of each pixel area at a lower cost.

[0063] Step 220 , after sorting the pixel regions whose region type is non-interference region based on the region area to obtain a sorting result, an area curve is constructed based on the sorting result.

[0064] Specifically, after determining the region type of each pixel region in the region image, the region area of ​​each pixel region whose region type is a non-interference region can be determined, and the pixel regions whose region type is a non-interference region can be sorted based on the region area to obtain a sorting result, that is, a sorting number of each pixel region can be obtained. Furthermore, the area of ​​each region can be arranged in a coordinate system with the horizontal axis being the sorting number and the vertical axis being the region area according to the sorting result, so as to construct an area curve according to the sorting result.

[0065] Figure 4 A schematic diagram of an area curve constructed according to the sorting results in another method for determining the stacking degree provided in an embodiment of the present invention, such as Figure 4 As shown in the figure, the black curve is the area curve constructed according to the sorting results.

[0066] In an embodiment of the present invention, after determining the area of ​​each pixel area whose area type is a non-interference area, the pixel areas whose area type is a non-interference area can be sorted based on the area area to obtain a sorting result, and then based on the sorting result, the area of ​​each area can be arranged in a coordinate system with the horizontal axis being the sorting number and the vertical axis being the area area, so as to construct an area curve according to the sorting result.

[0067] Step 230: Determine the area threshold in the area curve based on the elbow rule.

[0068] In one implementation, step 230 may specifically include:

[0069] The area curve is segmented by using each point in the area curve as a segmentation point to obtain a first area curve and a second area curve corresponding to each point; the first area curve and the second area curve corresponding to each point are linearly fitted to obtain a first straight line and a second straight line; a total fitting error corresponding to each point is determined according to a first fitting error between the first area curve and the first straight line corresponding to each point and a second fitting error between the second area curve and the second straight line; a target point is determined according to a minimum total fitting error, and the area corresponding to the target point in the area curve is determined as the area threshold.

[0070] Specifically, the area curve can be first divided into least squares fitting based on any point in the area curve, and then the two segments are linearly fitted based on the least squares method to obtain two straight lines. There is an error between the two straight lines obtained by fitting and the least squares fitting. All points in the area curve are traversed, and the point with the smallest error is determined as the target point, and then the area corresponding to the target point is determined as the area threshold.

[0071] Specifically, first, each point in the area curve can be used as a segmentation point to segment the area curve to obtain a first area curve and a second area curve corresponding to each point. Secondly, based on the least squares method, a linear fit can be performed on the first area curve corresponding to each point to obtain a first straight line. Based on the least squares method, a linear fit can be performed on the second area curve corresponding to each point to obtain a second straight line. Thirdly, a first fitting error between the first area curve and the first straight line corresponding to each point can be determined, and a second fitting error between the second area curve and the second straight line corresponding to each point can be determined. According to the first fitting error and the second fitting error, a total fitting error corresponding to each point can be determined. Then, the total fitting errors corresponding to each point can be compared, and the point corresponding to the minimum total fitting error can be determined as the target point, and the area corresponding to the target point can be determined as the area threshold.

[0072] like Figure 4 As shown, the two red lines are the first straight line and the second straight line obtained by linearly fitting the first area curve and the second area curve.

[0073] In an embodiment of the present invention, the target point with the smallest fitting error after segmenting the area curve and linearly fitting the segmentation results is determined in the area curve based on the elbow rule, and the area corresponding to the target point is determined as the area threshold, so as to determine the area threshold according to the area curve. The area threshold can be used to determine whether a pixel area with a region type of non-interference area is an object area, and provide data reference for determining the object area in each pixel area with a region type of non-interference area.

[0074] Step 240: Determine the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold.

[0075] Specifically, the area area and area threshold of each pixel area whose area type is non-interference area can be compared, and the pixel area whose area area is smaller than the area threshold among the pixel areas whose area type is non-interference area can be determined as the object area. These object areas can be used to determine the degree of object accumulation in the area to be identified.

[0076] In the embodiment of the present invention, the object area is determined in each pixel area whose area type is a non-interference area according to the area threshold.

[0077] Step 250: Determine the degree of accumulation of objects in the area to be identified according to the total area of ​​the object area and the area of ​​the area to be identified.

[0078] As mentioned above, the total area of ​​the article areas can be determined according to the area of ​​each article area, and the degree of accumulation of articles in the area to be identified can be determined according to the ratio of the total area of ​​the article areas to the area of ​​the area to be identified.

[0079] The total area of ​​the object region determined at this time can be understood as a pixel region whose area is smaller than the area threshold and does not contain a non-static object.

[0080] In the embodiment of the present invention, the degree of accumulation of objects in the area to be identified is determined according to the total area of ​​the object area and the area of ​​the area to be identified.

[0081] Step 260: When it is determined that the object accumulation degree is greater than the degree threshold, it is determined that the objects in the to-be-identified area are excessively accumulated.

[0082] The degree threshold may be pre-set according to actual needs and is not specifically limited here.

[0083] Specifically, the degree of object accumulation and the degree threshold are compared. When it is determined that the degree of object accumulation is greater than the degree threshold, it is determined that there is excessive accumulation of objects in the area to be identified. At this time, an early warning message can be sent to the terminal device to remind the staff that congestion has occurred in the area to be identified, so as to facilitate timely cleaning of the area to be identified and improve logistics efficiency.

[0084] In the embodiment of the present invention, by comparing the object accumulation degree with the degree threshold, when it is determined that the object accumulation degree is greater than the degree threshold, it is determined that the objects in the to-be-identified area are excessively accumulated, thereby realizing accumulation identification of the to-be-identified area.

[0085] The method for determining the degree of accumulation provided by an embodiment of the present invention includes: inputting a regional image of a to-be-identified area into a pre-trained segmentation model so that the segmentation model divides the regional image into a plurality of pixel areas and determines the area type of each of the pixel areas; after sorting the pixel areas whose area type is a non-interference area based on the area area to obtain a sorting result, constructing an area curve based on the sorting result; determining the area threshold in the area curve based on the elbow rule; determining the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold; determining the degree of object accumulation in the to-be-identified area according to the total area of ​​the object area and the area of ​​the to-be-identified area; and determining, when it is determined that the degree of object accumulation is greater than the degree threshold, that the objects in the to-be-identified area are excessively accumulated. In the above technical solution, after obtaining the regional image of the area to be identified, the regional image is input into a pre-trained segmentation model. The segmentation model can divide the regional image into multiple pixel areas according to the objects contained in the regional image, and can also determine the area type of each pixel area as an interference area or a non-interference area, so as to achieve the division of the regional image into multiple pixel areas and determine the area type of each pixel area at a relatively low cost, determine the area area of ​​each pixel area whose area type is a non-interference area, sort the pixel areas whose area type is a non-interference area based on the area area, obtain the sorting result, and arrange the area of ​​each area in a coordinate system whose horizontal axis is the sorting number and whose vertical axis is the area of ​​the area based on the sorting result, so as to achieve the construction of an area curve according to the sorting result, and based on the elbow method Then, in the area curve, a target point with the smallest fitting error is determined after segmenting the area curve and linearly fitting the segmentation results respectively, and the area corresponding to the target point is determined as the area threshold, so as to determine the area threshold according to the area curve, provide data reference for determining the object area in each pixel area whose area type is the non-interference area, and determine the object area in each pixel area whose area type is the non-interference area according to the area threshold. Compared with the prior art that determines the object area according to the target detection technology, the present application has a lower cost for determining the object area, and has higher accuracy and greater versatility. After determining the object area, the degree of object accumulation in the to-be-identified area can be determined according to the total area of ​​the object area and the area of ​​the to-be-identified area, so as to accurately determine the degree of object accumulation in the to-be-identified area.

[0086] In addition, by comparing the degree of accumulation of items and the degree threshold, when it is determined that the degree of accumulation of items is greater than the degree threshold, it is determined that there is excessive accumulation of items in the area to be identified, and accumulation identification of the area to be identified is realized, which is convenient for the staff to timely monitor the congestion in the area to be identified, and further facilitates the staff to clean up the area to be identified in time when congestion occurs in the area to be identified, thereby improving logistics efficiency.

[0087] Figure 5The schematic diagram of the structure of a device for determining the degree of accumulation provided by an embodiment of the present invention. The device and the methods for determining the degree of accumulation of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiments of the device for determining the degree of accumulation can refer to the embodiments of the methods for determining the degree of accumulation described above.

[0088] The specific structure of the stacking degree determination device is as follows: Figure 5 As shown, including:

[0089] An input module 510 is used to input a region image of a region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines a region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region;

[0090] A sorting module 520, configured to sort the pixel regions whose region type is a non-interference region based on the region area, and determine an area threshold according to the sorting result;

[0091] The determination module 530 is used to determine the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold, and determine the degree of object accumulation in the area to be identified according to the total area of ​​the object area and the area of ​​the area to be identified.

[0092] Based on the above embodiment, the device further includes:

[0093] The execution module is used to determine that the objects in the to-be-identified area are excessively accumulated when it is determined that the accumulation degree of the objects is greater than a degree threshold.

[0094] Based on the above embodiment, the sorting module 520 is specifically used for:

[0095] After sorting the pixel regions whose region type is non-interference region based on the region area to obtain a sorting result, an area curve is constructed based on the sorting result; and the area threshold is determined in the area curve based on the elbow rule.

[0096] In one implementation, determining the area threshold in the area curve based on the elbow rule includes:

[0097] The area curve is segmented by using each point in the area curve as a segmentation point to obtain a first area curve and a second area curve corresponding to each point; the first area curve and the second area curve corresponding to each point are linearly fitted to obtain a first straight line and a second straight line; a total fitting error corresponding to each point is determined according to a first fitting error between the first area curve and the first straight line corresponding to each point and a second fitting error between the second area curve and the second straight line; a target point is determined according to a minimum total fitting error, and the area corresponding to the target point in the area curve is determined as the area threshold.

[0098] Based on the above embodiment, the determination module 530 is specifically used for:

[0099] The pixel areas whose area types are non-interference areas and whose area areas are smaller than the area threshold are determined as the object areas; and the area ratio of the total area of ​​the object areas to the area of ​​the area to be identified is determined as the degree of object accumulation in the area to be identified.

[0100] Based on the above embodiment, the device further includes:

[0101] A training module is used to obtain a training image, process the training image based on an initial segmentation model, and obtain a region vector of each segmentation region corresponding to the training image; determine the region type of each segmentation region corresponding to each training image; use the region vector of each segmentation region corresponding to each training image as a training input of a classification module in the initial segmentation model, and use the region type of each segmentation region as a training output to guide the training output to perform network training on the classification module to obtain the segmentation model, wherein the initial segmentation model is composed of the classification module and the initial segmentation model.

[0102] The device for determining the degree of accumulation provided in the embodiment of the present invention can execute the method for determining the degree of accumulation provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method for determining the degree of accumulation.

[0103] It is worth noting that in the embodiment of the above-mentioned stacking degree determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0104] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 A block diagram of an exemplary electronic device 6 suitable for implementing embodiments of the present invention is shown. Figure 6The electronic device 6 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0105] like Figure 6 As shown, the electronic device 6 is in the form of a general purpose computing electronic device. The components of the electronic device 6 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0106] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0107] The electronic device 6 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 6, including volatile and non-volatile media, removable and non-removable media.

[0108] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 6 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0109] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0110] The electronic device 6 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the electronic device 6, and / or any device that enables the electronic device 6 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 6 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. Figure 6 As shown, the network adapter 20 communicates with other modules of the electronic device 6 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 6, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0111] The processing unit 16 executes various functional applications and page displays by running the program stored in the system memory 28, for example, implementing the method for determining the degree of accumulation provided by the embodiment of the present invention, which includes:

[0112] Inputting a region image of the region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region;

[0113] Sorting the pixel regions whose region types are non-interference regions based on the region areas, and determining the area threshold according to the sorting result;

[0114] An object area is determined in each of the pixel areas whose area type is a non-interference area according to the area threshold, and an object accumulation degree of the area to be identified is determined according to a total area of ​​the object area and an area of ​​the area to be identified.

[0115] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the method for determining the degree of accumulation provided by any embodiment of the present invention.

[0116] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a method for determining a pile-up degree provided in an embodiment of the present invention is implemented. The method includes:

[0117] Inputting a region image of the region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region;

[0118] Sorting the pixel regions whose region types are non-interference regions based on the region areas, and determining the area threshold according to the sorting result;

[0119] An object area is determined in each of the pixel areas whose area type is a non-interference area according to the area threshold, and an object accumulation degree of the area to be identified is determined according to a total area of ​​the object area and an area of ​​the area to be identified.

[0120] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0121] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0122] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0123] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0125] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for determining a stacking degree, characterized in that: include: Inputting a region image of the region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region; Sorting the pixel regions whose region types are non-interference regions based on the region areas, and determining the area threshold according to the sorting result; An object area is determined in each of the pixel areas whose area type is a non-interference area according to the area threshold, and an object accumulation degree of the area to be identified is determined according to a total area of ​​the object area and an area of ​​the area to be identified.

2. The method for determining the degree of accumulation according to claim 1, characterized in that: After determining the degree of accumulation of objects in the to-be-identified area according to the total area of ​​the object area and the area of ​​the to-be-identified area, the method further includes: When it is determined that the object accumulation degree is greater than the degree threshold, it is determined that the objects in the to-be-identified area are excessively accumulated.

3. The method for determining the degree of accumulation according to claim 1, characterized in that: Sorting the pixel regions whose region types are non-interference regions based on the region areas, and determining the area threshold according to the sorting results, including: After sorting the pixel regions whose region types are non-interference regions based on the region areas to obtain sorting results, an area curve is constructed based on the sorting results; The area threshold is determined in the area curve based on the elbow rule.

4. The method for determining the degree of accumulation according to claim 3, characterized in that: Determining the area threshold in the area curve based on the elbow rule includes: Using each point in the area curve as a segmentation point to segment the area curve, to obtain a first area curve and a second area curve corresponding to each point; Performing linear fitting on the first area curve and the second area curve corresponding to each point to obtain a first straight line and a second straight line; Determine a total fitting error corresponding to each point according to a first fitting error between the first area curve and the first straight line and a second fitting error between the second area curve and the second straight line corresponding to each point; A target point is determined according to the minimum total fitting error, and an area corresponding to the target point in the area curve is determined as the area threshold.

5. The method for determining the accumulation degree according to claim 1, characterized in that: Determining the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold includes: The pixel region whose region area is smaller than the area threshold among the pixel regions whose region type is a non-interference region is determined as the object region.

6. The method for determining the accumulation degree according to claim 1, characterized in that: Determining the degree of accumulation of objects in the to-be-identified area according to the total area of ​​the object area and the area of ​​the to-be-identified area includes: The area ratio of the total area of ​​the article area to the area of ​​the to-be-identified area is determined as the degree of article accumulation in the to-be-identified area.

7. The method for determining the accumulation degree according to claim 1, characterized in that: The training process of the partitioning model includes: Acquire a training image, and process the training image based on the segmentation model to obtain a region vector of each segmented region corresponding to the training image; Determining a region type of each of the segmented regions corresponding to each of the training images; The classification module is trained on the network using the region vector of each segmented region corresponding to each training image as the training input of the classification module in the initial segmentation model and the region type of each segmented region as the guidance of the training output to obtain the segmentation model, wherein the initial segmentation model is composed of the classification module and the segmentation model.

8. A device for determining a stacking degree, characterized in that: include: An input module, used for inputting a region image of a region to be identified into a pre-trained segmentation model, so that the segmentation model divides the region image into a plurality of pixel regions and determines the region type of each pixel region, wherein the segmentation model is composed of a classification module and a segmentation model, and the region type is an interference region or a non-interference region; A sorting module, used for sorting the pixel regions whose region type is a non-interference region based on the region area, and determining the area threshold according to the sorting result; The determination module is used to determine the object area in each of the pixel areas whose area type is a non-interference area according to the area threshold, and determine the degree of object accumulation in the area to be identified according to the total area of ​​the object area and the area of ​​the area to be identified.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the degree of accumulation as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the method for determining the degree of accumulation as claimed in any one of claims 1 to 7 when executed by a computer processor.

Images