Object processing method and device, computer equipment and readable storage medium

Abstract

The invention discloses an object processing method and device, computer equipment and a readable storage medium. The object processing method comprises the steps of obtaining size information of each detection frame output by a detector; wherein each detection frame contains an object detected by the detector in the target image; based on the size information of each detection frame, determining the size density state of all detected objects in the target image; wherein the size density state is used for representing that the sizes of all detected objects are uniform or non-uniform; and clustering all detected objects in the target image based on a clustering algorithm associated with the size density state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data recognition, and in particular to an object processing method and device, computer equipment and a readable storage medium. BACKGROUND

[0002] When the image contains multiple objects (also known as multiple clusters of objects or multiple groups of objects), the prior art only uses a single clustering algorithm to implement object clustering, which cannot guarantee clustering accuracy. In particular, in the field of architecture, when the objects contained in the image are multiple piles of building materials, since the number of piles is unknown, the shapes of the piles are mostly irregular, and the sizes of the building materials in some piles are different, clustering based on a single clustering algorithm in the prior art has poor clustering effect.

[0003] In view of the above problems of the prior art, there is currently no effective solution. SUMMARY

[0004] The present application aims to provide an object processing method and device, computer equipment and a readable storage medium, which can solve the defect that the prior art cannot guarantee clustering accuracy using a single clustering algorithm.

[0005] According to one aspect of the present application, an object processing method is provided, comprising: obtaining size information of each detection frame output by a detector; wherein each detection frame contains an object detected by the detector in a target image; determining a size density state of all detected objects in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the sizes of all detected objects are uniform or non-uniform; clustering all detected objects in the target image based on a clustering algorithm associated with the size density state.

[0006] Optionally, determining the size density state of all detected objects in the target image based on the size information of each detection frame comprises: obtaining the height and width of all detection frames from all size information as a group of data, and calculating the variance of the group of data; when the variance is less than or equal to a preset variance threshold, determining that the size density state is used to represent that the sizes of all detected objects are uniform; when the variance is greater than the preset variance threshold, determining that the size density state is used to represent that the sizes of all detected objects are non-uniform.

[0007] Optionally, when the size density state is used to represent that sizes of all detected objects are uniform, the clustering of all detected objects in the target image based on the clustering algorithm associated with the size density state comprises: obtaining the height and width of all bounding boxes from all size information as a group of data, and calculating the average value of the group of data; obtaining a preset neighborhood radius weight, weighting the average value by using the neighborhood radius weight to obtain an average weighted value; wherein the average weighted value is greater than the average value and less than or equal to twice the average value; taking the average weighted value as the neighborhood radius in the DBSCAN algorithm, and calling the DBSCAN algorithm to cluster the bounding boxes of all detected objects.

[0008] Optionally, when the size density state is used to represent that sizes of all detected objects are not uniform, the clustering of all detected objects in the target image based on the clustering algorithm associated with the size density state comprises: calculating the Euclidean distance of each two bounding boxes based on the center coordinates of the bounding boxes in all size information; calculating the size difference of each two bounding boxes based on the height and width of the bounding boxes in all size information; taking the Euclidean distance and the size difference of each two bounding boxes as input parameters of a preset size perception distance formula to calculate the size perception distance of each two bounding boxes; constructing an N-dimensional matrix based on the size perception distance of each two bounding boxes; wherein N is the number of bounding boxes, and the size perception distance on the main diagonal line in the N-dimensional matrix is 0; taking the N-dimensional matrix as a distance matrix in the OPTICS algorithm, and calling the OPTICS algorithm to cluster the bounding boxes of all detected objects.

[0009] Optionally, the calculation of the size perception distance of each two bounding boxes by taking the Euclidean distance and the size difference of each two bounding boxes as input parameters of a preset size perception distance formula comprises: obtaining a preset Euclidean distance weight and a size difference weight; wherein the Euclidean distance weight is less than the size difference weight; weighting the Euclidean distance of each two bounding boxes by using the Euclidean distance weight to obtain an Euclidean distance weighted value of each two bounding boxes; weighting the size difference of each two bounding boxes by using the size difference weight to obtain a size difference weighted value of each two bounding boxes; calculating the size perception distance of each two bounding boxes based on the Euclidean distance weighted value and the size difference weighted value of each two bounding boxes.

[0010] Optionally, after the clustering of all the detected objects in the target image based on the clustering algorithm associated with the size density state, the object processing method further comprises: filtering the size information of the detection frame of each cluster object from all the size information; obtaining all the corner point coordinates contained in the size information of the detection frame of each cluster object; performing convex hull calculation on all the corner point coordinates corresponding to each cluster object to obtain the convex hull vertex coordinates corresponding to each cluster object; generating a semi-transparent mask for covering each cluster object based on the convex hull vertex coordinates corresponding to each cluster object.

[0011] Optionally, after the generation of the semi-transparent mask for covering each cluster object based on the convex hull vertex coordinates corresponding to each cluster object, the object processing method further comprises: in response to a mask selection instruction, determining a target semi-transparent mask selected by the mask selection instruction; calculating the overlap ratio of each detection frame and the target semi-transparent mask; wherein the overlap ratio is the ratio between the overlapping area of the detection frame and the target semi-transparent mask and the area of the detection frame; counting the number of detection frames whose overlap ratio is greater than or equal to a preset ratio threshold value; taking the counted number as the number of objects covered by the target semi-transparent mask.

[0012] In order to achieve the above-mentioned purpose, the present application further provides an object processing device, comprising: an acquisition module configured to acquire the size information of each detection frame output by a detector; wherein each detection frame contains an object detected by the detector in a target image; a determination module configured to determine the size density state of all the detected objects in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the sizes of all the detected objects are uniform or non-uniform; a clustering module configured to cluster all the detected objects in the target image based on a clustering algorithm associated with the size density state.

[0013] In order to achieve the above-mentioned purpose, the present application further provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor is configured to implement the steps of the above-mentioned object processing method when executing the computer program.

[0014] In order to achieve the above object, the application further provides a computer readable storage medium, which stores a computer program, and the computer program is used to realize the steps of the object processing method.

[0015] The object processing method, device, computer equipment and readable storage medium provided by the application can determine whether the sizes of all the detected objects are uniform through the size information of all the detection boxes, and further dynamically select a corresponding clustering algorithm for clustering based on the uniformity of the sizes of the detected objects. The embodiment dynamically selects a clustering algorithm based on the uniformity of the sizes of all the objects in the image, which can well guarantee the accuracy of the clustering result and has universality. BRIEF DESCRIPTION OF DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Moreover, the same reference numerals are intended to denote the same components throughout the accompanying drawings. In the drawings: Figure 1 A flowchart of the object processing method provided for Embodiment One; Figure 2 A schematic diagram of the object clustering and counting scheme provided for Embodiment One; Figure 3 A block diagram of the object processing device provided for Embodiment Two; Figure 4 A block diagram of the computer equipment suitable for realizing the object processing method provided for Embodiment Three. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0018] Embodiment One The embodiment of the application provides an object processing method, as shown in the figure, the method comprises steps S1-S3, wherein: Figure 1 S1, obtaining size information of each detection box output by a detector; wherein each detection box contains an object detected by the detector in a target image. S1, obtaining size information of each detection box output by a detector; wherein each detection box contains an object detected by the detector in a target image.

[0019] The detector is used to identify objects in the target image. After the target image is input into the detector, the detector draws a detection box for marking the objects on the target image to achieve the identification effect. The detector can be a YOLO model, such as YOLOV4, YOLOV5, etc.

[0020] During the process of identifying the target image based on the detector, the inventors consider that when the number of objects in the target image is too large and too dense, the resolution of a single object is too small, and for the detection of such small target objects, it is easy to miss detection and false detection. For example, the object is building materials, and when the resolution of the cross section of a single material is too small, the detector is used to identify the building materials, which is easy to miss detection and false detection. To solve this problem, the inventors think of using slice reasoning technology to increase the resolution of small target objects. Specifically as follows: The user first selects the object type to be detected on the operation interface, and then takes or uploads the target image of the corresponding type to the object recognition operation interface through a mobile terminal (such as a mobile phone). At this time, the system underlying layer will perform target detection-based slice reasoning on the image. Since the YOLO model has high real-time performance, good detection effect, and easy deployment, the system underlying layer uses a YOLO model optimized by training to perform target detection on the image.

[0021] (1) Slice the image A. Determination of the number of slices According to different intervals of the width and height resolution of the target image, the number of times the width and height need to be cut is dynamically selected, and then the number of slices is obtained. For example: the width and height of the image are w and h respectively, and assuming that according to the preset interval, the width w of the image needs to be cut 2 times, and the height h of the image needs to be cut 2 times, then the number of slices is 2x2=4.

[0022] B. Determination of the size of the slice Since direct slicing and close cutting between slices cannot avoid the complete object (such as the cross section of the material) being divided into two adjacent slices, which in turn leads to multiple detection in the subsequent process. Therefore, in order to ensure that each object is at least complete in one slice, there needs to be a certain overlap between adjacent slices.

[0023] The width and height of the overlap can be preset to 0.1 to 0.3 of the original slice width and height, and the specific value can be further confirmed by testing, and thus the size of each final slice can be determined. For example: the width and height of the original slice are 0.5w and 0.5h respectively, and assuming that the original width and height ratio of the overlap is 0.2 of the slice, then the final size of the slice is 0.6w and 0.6h.

[0024] (2) Reasoning on all slice subgraphs After the resize and other processing of all slice subgraphs, they are respectively input into the trained and optimized yolo target detector to perform inference and obtain the detection box results of each subgraph after inference. In order to reduce the inference time, batch inference can be performed on the slice subgraphs.

[0025] Since the resolution of the slice is smaller than that of the original image, when the slice is resized, the pixel value of the small target in the slice will be larger than that of the original image after resizing. Suppose: Before resizing: Original image resolution: 2000*2000 pixels Slice resolution: 1000*1000 pixels Input resolution of yolo target detection model: 640*640 pixels Resolution of small target in original image: 20*20 pixels After resizing: The resolution of the small target after resizing of the original image is 6.4*6.4 pixels The resolution of the small target after resizing of the slice subgraph is 12.8*12.8 pixels Therefore, the magnification of the small target is (12.8*12.8) / (6.4*6.4)=4 times Therefore, the small target in the slice will be enlarged several times after resizing compared with that in the original image after resizing; at the same time, since each slice only contains a local area of the original image, the context noise can be reduced to a certain extent. Therefore, the slice inference method can reduce the false detection and missed detection of small targets.

[0026] The size information includes the coordinates of the four corners of the detection box, the height and width of the detection box.

[0027] (3) Merging and deduplication of slice subgraph detection results First, the coordinates of the detection box results of each slice subgraph are converted and remapped back to the original image; then the detection box results of all slice subgraphs are merged and stored in the same list; then the list storing all detection box results is deduplicated, that is, the merging and deduplication of all overlapping area detection boxes are completed; finally, the detection box results of the whole image are obtained.

[0028] In step S2, the size density state of all objects detected in the target image is determined based on the size information of each detection box; wherein the size density state is used to represent whether the size of all detected objects is uniform or non-uniform.

[0029] An alternative solution for determining the size density state is to compare the area difference of each two bounding boxes one by one, to count a first number of bounding boxes whose area difference is less than or equal to a preset area difference threshold, to calculate a first ratio of the first number to the total number of bounding boxes output by the detector, and to determine that the size density state is used to represent that the sizes of all the detected objects are uniform if the first ratio is greater than a first preset value, otherwise to determine that the size density state is used to represent that the sizes of all the detected objects are not uniform.

[0030] An alternative solution for determining the size density state is to compare the height difference and the width difference of each two bounding boxes one by one, to count a second number of bounding boxes whose height difference is less than or equal to a preset height difference threshold, to count a third number of bounding boxes whose width difference is less than or equal to a preset height difference threshold, to calculate a second ratio of the second number to the total number of bounding boxes output by the detector, to calculate a third ratio of the third number to the total number of bounding boxes output by the detector, and to determine that the size density state is used to represent that the sizes of all the detected objects are uniform if the second ratio and / or the third ratio is greater than a second preset value, otherwise to determine that the size density state is used to represent that the sizes of all the detected objects are not uniform.

[0031] The inventor considers that the core role of variance is to quantify the deviation of data from the mean value, to measure the dispersion degree of data, to evaluate the risk, and to serve as the basis for statistical inference, and therefore further proposes another alternative solution for determining the size density state. Specifically, step S2 includes: obtaining the height and the width of all the bounding boxes from all the size information as a group of data, and calculating the variance of the group of data; determining that the size density state is used to represent that the sizes of all the detected objects are uniform when the variance is less than or equal to a preset variance threshold; determining that the size density state is used to represent that the sizes of all the detected objects are not uniform when the variance is greater than the preset variance threshold.

[0032] All size information is size information of all detection boxes output by the detector. In this embodiment, the width and height of all detection boxes are first obtained, and then a variance is calculated by taking the width and height together. If the variance is less than or equal to a preset variance threshold, it indicates that the sizes of all objects on the current image are uniform, and the difference is not large, and at this time, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm which can process density uniform data and is more efficient and simple is used; if the variance is greater than the preset variance threshold, it indicates that the sizes of all objects on the current image are not uniform, and the difference is large, and at this time, the OPTICS (Ordering Points To Identify the Clustering Structure) algorithm which can process density non-uniform data is used.

[0033] In step S3, all objects detected in the target image are clustered based on the clustering algorithm associated with the size density state.

[0034] In view of the unknown number of piles in the image shot or uploaded by the user, the irregular shape of the piles, and in order to ensure the efficiency of clustering calculation and the clustering effect of objects of different sizes, the system underlying layer adopts a density clustering algorithm to cluster the objects. That is, the clustering algorithm associated with the size density state is a density clustering algorithm.

[0035] When the size density state is used to represent that the sizes of all detected objects are uniform, the clustering algorithm associated with the size density state is the DBSCAN algorithm; when the size density state is used to represent that the sizes of all detected objects are not uniform, the clustering algorithm associated with the size density state is the OPTICS algorithm.

[0036] Optionally, when the size density state is used to represent that the sizes of all detected objects are uniform, step S3 includes: obtaining the height and width of all detection boxes from all size information as a group of data, and calculating the average value of the group of data; obtaining a preset neighborhood radius weight, weighting the average value by using the neighborhood radius weight to obtain an average weighted value; wherein the average weighted value is greater than the average value and less than or equal to twice the average value; taking the average weighted value as the neighborhood radius in the DBSCAN algorithm, and calling the DBSCAN algorithm to cluster the detection boxes of all detected objects.

[0037] The neighborhood radius is used to determine whether two points are adjacent to each other. For example, if the distance between two points is less than or equal to the neighborhood radius, they are considered to be adjacent to each other, and it is possible to divide them into the same cluster. In the prior art, when setting the neighborhood radius of the DBSCAN algorithm, a fixed value is usually preset. This static setting of the neighborhood radius can easily identify multiple piles of objects as the same pile. To solve this problem, the present application creatively proposes to dynamically set the neighborhood radius based on the size of the detection box. Specifically, without distinguishing between the height and width of the detection box, the values representing the height and the values representing the width are all put into the same data group as the data to be processed, and then the average value of the group data is calculated. Further, the average value is weighted so that the average weighted value obtained after weighting is greater than the calculated average value and less than or equal to twice the average value. Since the object size in the current image is uniform, using the neighborhood radius setting method of the present application can determine whether the current object and the next object belong to the same cluster one by one, i.e., there is no problem of missing judgment, and other pile objects will not be attributed to the current pile object.

[0038] Further, according to actual needs, a number is given as the value of the minimum neighbor number (min_samples), which can be 3 to 5, indicating that at least min_samples objects are needed to form a valid cluster; then the DBSCAN algorithm is called, and the neighborhood radius and min_samples (if necessary, other parameter values can also be adjusted) are calculated to perform clustering calculation on the detection box, and objects that meet the parameter conditions are clustered into a pile.

[0039] Alternatively, when the size density state is used to represent that the sizes of all detected objects are not uniform, step S3 includes: Based on the center coordinates of the detection boxes in all size information, the Euclidean distance between each two detection boxes is calculated; Based on the height and width of the detection boxes in all size information, the size difference between each two detection boxes is calculated; The Euclidean distance and the size difference between each two detection boxes are taken as input parameters of a preset size-aware distance formula to calculate the size-aware distance between each two detection boxes; Based on the size-aware distance between each two detection boxes, an N-dimensional matrix is constructed; where N is the number of detection boxes, and the size-aware distances on the main diagonal of the N-dimensional matrix are all 0; The N-dimensional matrix is taken as the distance matrix in the OPTICS algorithm, and the OPTICS algorithm is called to cluster the detection boxes of all detected objects.

[0040] The key to processing objects with different sizes using the OPTICS algorithm is to define a size-aware distance matrix. Because the original distance matrix in the algorithm only considers the Euclidean distance between the center points and does not consider the size information of the objects, objects with different sizes may be incorrectly clustered when they are adjacent in space. Therefore, a size-aware distance matrix is calculated by combining the spatial Euclidean distance and size information, which separates objects with different sizes, promotes the clustering of objects with similar sizes, and improves the robustness of clustering. The detailed steps are as follows: Taking the center coordinates of the bounding box as the positioning point of the bounding box, the Euclidean distance between each two bounding boxes is calculated. Taking the size difference between two bounding boxes as an example, the scheme for calculating the size difference between each two bounding boxes is as follows: calculate the absolute value of the difference between the heights of the two bounding boxes and the absolute value of the difference between the widths, calculate the ratio of the two absolute values as the size difference between the two bounding boxes, or calculate the product of the two absolute values as the size difference between the two bounding boxes, or calculate the sum of the two absolute values as the size difference between the two bounding boxes. Another optional scheme is: calculate the areas of the two bounding boxes based on height and width, calculate the area difference between the two bounding boxes as the size difference between the two bounding boxes, or calculate the area ratio of the two bounding boxes as the size difference between the two bounding boxes. It should be noted that any calculation scheme that can reflect the size difference between two bounding boxes is included in the scheme of the present embodiment.

[0041] Further, the size-aware distance between each two bounding boxes is calculated. Since the size-aware distance includes both the Euclidean distance between the two bounding boxes and the size difference between the two bounding boxes, the N-dimensional matrix constructed based on the size-aware distance fully considers the size information of the objects. When the N-dimensional matrix is used as the distance matrix in the OPTICS algorithm for clustering calculation, the spatial and size information are combined, which separates objects with different sizes, promotes the clustering of objects with similar sizes, and improves the robustness of clustering.

[0042] Alternatively, the Euclidean distance and size difference between each two bounding boxes are taken as input parameters of a preset size-aware distance formula to calculate the size-aware distance between each two bounding boxes, which includes: Obtaining a preset Euclidean distance weight and a size difference weight; wherein the Euclidean distance weight is less than the size difference weight; Weighting the Euclidean distance between each two bounding boxes using the Euclidean distance weight to obtain the Euclidean distance weighted value of each two bounding boxes; Weighting the size difference between each two bounding boxes using the size difference weight to obtain the size difference weighted value of each two bounding boxes; Based on the Euclidean distance weighted value and the size difference weighted value of each two bounding boxes, the size-aware distance between each two bounding boxes is calculated.

[0043] An optional solution is that the sum of the Euclidean distance weight and the size difference weight is 1, or the sum of the Euclidean distance weight and the size difference weight is not limited, and the Euclidean distance weight is set to be slightly smaller and the size difference weight is set to be slightly larger to ensure that the size difference has sufficient influence. Preferably, the size difference weight is greater than 2 times the Euclidean distance weight.

[0044] Taking the calculation of the size perception distance of two detection boxes as an example, the scheme for calculating the size perception distance of each two detection boxes is explained: calculating the sum of the Euclidean distance weighted value and the size difference weighted value of the two detection boxes as the size perception distance of each two detection boxes; or calculating the product of the Euclidean distance weighted value and the size difference weighted value of the two detection boxes as the size perception distance of each two detection boxes; or calculating the ratio of the Euclidean distance weighted value and the size difference weighted value of the two detection boxes as the size perception distance of each two detection boxes. Preferably, in order to strengthen the influence of the size difference weighted value, the sum of the Euclidean distance weighted value and the size difference weighted value of the two detection boxes is taken as the size perception distance of each two detection boxes.

[0045] Optionally, after clustering all the detected objects in the target image based on the clustering algorithm associated with the size density state, the object processing method further comprises: Filtering the size information of the detection box of each cluster object from all the size information; Obtaining all the corner point coordinates contained in the size information of the detection box of each cluster object; Performing convex hull calculation on all the corner point coordinates corresponding to each cluster object to obtain the convex hull vertex coordinates corresponding to each cluster object; Generating a semi-transparent mask for covering each cluster object based on the convex hull vertex coordinates corresponding to each cluster object.

[0046] The embodiment is used to generate a semi-transparent mask for each cluster object respectively, so that when a certain cluster object is selected, manual selection is no longer needed as in the prior art, but the object cluster covered by the corresponding semi-transparent mask can be directly selected by clicking the semi-transparent mask, which is accurate and convenient.

[0047] First, the four corner point coordinates of all the detection boxes of each cluster object after clustering are obtained, and it is noted that the center point coordinates are not obtained, and then the four corner point coordinates are stored in the same list; the ConvexHull convex hull calculation is performed on the corner point coordinate list to obtain the convex hull vertex coordinates; then, according to the convex hull vertex coordinates, an irregular semi-transparent mask covering each cluster object is drawn and filled on the original image on the operation interface.

[0048] In some special cases, some objects to be detected may not be clustered, or some objects not to be detected may be wrongly clustered. In this case, the user can adjust the generated semi-transparent mask on the operation interface according to the actual situation.

[0049] Optionally, after the semi-transparent mask for covering each cluster of objects is generated based on the convex hull vertex coordinates corresponding to each cluster of objects, the object processing method further comprises: In response to a mask selection instruction, determining a target semi-transparent mask selected by the mask selection instruction; calculating an overlap ratio of each detection frame and the target semi-transparent mask; wherein the overlap ratio is a ratio between an overlapping area of the detection frame and the target semi-transparent mask and an area of the detection frame; counting a number of detection frames whose overlap ratios are greater than or equal to a preset ratio threshold value; taking the counted number as a number of objects covered by the target semi-transparent mask.

[0050] The user does not need to manually select the object pile to be detected on the operation interface, but can directly select a certain semi-transparent mask to obtain the number of the corresponding object pile covered by the mask. If the user does not select the mask, the interface displays the number of all objects in the image by default. Therefore, the user can flexibly select the number of each object pile in the image and the number of all objects in the image.

[0051] The logic of obtaining the number of objects in the object pile covered by the mask is as follows: all detection frames are looped and traversed, the overlap ratio of each detection frame and the current mask is calculated (overlap ratio = area of overlapping part of the detection frame and the mask / detection frame area), and if the overlap ratio exceeds the set threshold value, the count is increased by 1. Finally, the total count obtained after the traversal is completed is the number of objects in the object pile covered by the mask.

[0052] The density clustering algorithm used in the present application can solve the following two problems: a. In actual scenes, since objects of the same type are mostly stacked together in a designated area, there are inevitably some other stacks of objects of the same type that do not need to be recognized at the edge position of the image to be detected by the user. In this case, the user needs to manually select the object pile area to be detected to exclude other objects of the same type that do not need to be recognized, and then obtain the number of objects in the object pile to be detected; b. The problem of counting multiple object piles, especially multiple object piles of different sizes. Based on the present application, the detection frames are clustered to achieve: a. automatically generating a semi-transparent mask corresponding to each object pile, so that the user does not need to manually select the object pile area for counting; b. counting multiple object piles, especially multiple object piles of different sizes, so that the user can take a picture for detection at one time and obtain the respective numbers of multiple object piles.

[0053] The present application takes the object in the picture as an example to illustrate the detailed process of the present application. As shown in Figure 2 The user shoots or uploads a material image to be detected, and performs slicing processing on the image, and the slices have repeated parts; then the slice sub-image is input into the pre-trained yolo model to perform inference to obtain the detection frame on each slice sub-image, and then the inference result is de-duplicated to obtain the detection frame of all materials detected in the original material image. According to the size information of all detection frames, it is judged whether the size of the materials in the image is uniform, and based on the uniformity, a corresponding density clustering algorithm is selected to cluster the materials into piles. Each pile of clustered materials is subjected to convex hull processing, and a semi-transparent mask of each pile of materials is automatically generated, and manual adjustment of the mask is supported. The user selects a semi-transparent mask to obtain the number of materials in the material pile covered by the mask; if not selected, the number of all materials in the original material image is displayed by default.

[0054] Embodiment two The object processing device provided by the embodiment of the present application is as shown in Figure 3 The object processing device 30 specifically includes the following components: The acquisition module 301 is configured to acquire size information of each detection frame output by a detector; wherein each detection frame contains an object in a target image detected by the detector; The determination module 302 is configured to determine a size density state of all objects detected in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the size of all detected objects is uniform or not uniform; The clustering module 303 is configured to cluster all objects detected in the target image based on a clustering algorithm associated with the size density state.

[0055] Optionally, the determination module is specifically configured to: acquire the height and width of all detection frames from all size information as a group of data, and calculate the variance of the group of data; When the variance is less than or equal to a preset variance threshold, it is determined that the size density state is used to represent that the size of all detected objects is uniform; When the variance is greater than the preset variance threshold, it is determined that the size density state is used to represent that the size of all detected objects is not uniform.

[0056] Optionally, when the size density state is used to represent that the size of all detected objects is uniform, the clustering module is specifically configured to: acquire the height and width of all detection frames from all size information as a group of data, and calculate the average value of the group of data; obtain a preset neighborhood radius weight, and weight the average value by using the neighborhood radius weight to obtain an average weighted value; wherein the average weighted value is greater than the average value and less than or equal to twice the average value; use the average weighted value as a neighborhood radius in a DBSCAN algorithm, and call the DBSCAN algorithm to cluster the bounding boxes of all the detected objects.

[0057] Optionally, when the size density state is used to represent that the sizes of all the detected objects are uneven, the clustering module is specifically configured to: calculate the Euclidean distance between each two bounding boxes based on the center coordinates of the bounding boxes in all the size information; calculate the size difference between each two bounding boxes based on the height and width of the bounding boxes in all the size information; use the Euclidean distance and the size difference between each two bounding boxes as input parameters of a preset size-aware distance formula to calculate the size-aware distance between each two bounding boxes; construct an N-dimensional matrix based on the size-aware distance between each two bounding boxes; wherein N is the number of the bounding boxes, and the size-aware distance on the main diagonal line in the N-dimensional matrix is 0; use the N-dimensional matrix as a distance matrix in an OPTICS algorithm, and call the OPTICS algorithm to cluster the bounding boxes of all the detected objects.

[0058] Optionally, when the clustering module performs the step of using the Euclidean distance and the size difference between each two bounding boxes as input parameters of a preset size-aware distance formula to calculate the size-aware distance between each two bounding boxes, the clustering module is specifically configured to: obtain a preset Euclidean distance weight and a size difference weight; wherein the Euclidean distance weight is less than the size difference weight; weight the Euclidean distance between each two bounding boxes by using the Euclidean distance weight to obtain an Euclidean distance weighted value of each two bounding boxes; weight the size difference between each two bounding boxes by using the size difference weight to obtain a size difference weighted value of each two bounding boxes; calculate the size-aware distance between each two bounding boxes based on the Euclidean distance weighted value and the size difference weighted value of each two bounding boxes.

[0059] Optionally, the object processing apparatus further comprises: a screening module configured to, after clustering all the detected objects in the target image based on the clustering algorithm associated with the size density state, screen the size information of the bounding boxes of each cluster of objects from all the size information; The extraction module is configured to acquire all corner point coordinates contained in size information of the detection frame of each cluster of objects. The first calculation module is configured to perform convex hull calculation on all corner point coordinates corresponding to each cluster of objects to obtain convex hull vertex coordinates corresponding to each cluster of objects. The generation module is configured to generate a semi-transparent mask for covering each cluster of objects based on the convex hull vertex coordinates corresponding to each cluster of objects.

[0060] Optionally, the object processing apparatus further includes: The first processing module is configured to, after the semi-transparent mask for covering each cluster of objects is generated based on the convex hull vertex coordinates corresponding to each cluster of objects, determine a target semi-transparent mask selected by a mask selection instruction in response to the mask selection instruction. The second calculation module is configured to calculate an overlap ratio of each detection frame and the target semi-transparent mask, wherein the overlap ratio is a ratio between an overlapping area of the detection frame and the target semi-transparent mask and an area of the detection frame. The statistical module is configured to count a number of detection frames with an overlap ratio greater than or equal to a preset ratio threshold. The second processing module is configured to take the counted number as a number of objects covered by the target semi-transparent mask.

[0061] Embodiment Three The embodiment also provides a computer device, such as a smartphone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a cabinet server (including a single server or a server cluster composed of multiple servers), which can execute a program. As shown in the figure, the computer device 40 of the embodiment at least includes but is not limited to a memory 401 and a processor 402 which can be connected to each other through a system bus. Figure 4 It should be noted that, Figure 4 Only the computer device 40 with the components 401-402 is shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0062] In this embodiment, the memory 401 (i.e., a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 401 can be an internal storage unit of the computer device 40, such as a hard disk or a memory of the computer device 40. In other embodiments, the memory 401 can also be an external storage device of the computer device 40, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 40. Of course, the memory 401 can also include both the internal storage unit and the external storage device of the computer device 40. In this embodiment, the memory 401 is generally used to store an operating system and various application software installed on the computer device 40. In addition, the memory 401 can also be used to temporarily store various data that have been output or will be output.

[0063] The processor 402 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 402 is generally used to control the overall operation of the computer device 40.

[0064] Specifically, in this embodiment, the processor 402 is configured to execute a program of an object processing method stored in the memory 401, and the program of the object processing method, when executed, implements the following steps: obtain size information of each detection frame output by a detector; wherein each detection frame contains an object detected by the detector in a target image; determine a size density state of all the detected objects in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the sizes of all the detected objects are uniform or non-uniform; perform clustering on all the detected objects in the target image based on a clustering algorithm associated with the size density state.

[0065] The specific implementation process of the above method steps can be referred to Embodiment One, which will not be repeated here.

[0066] Embodiment Four The embodiment also provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application market, and the like, which stores a computer program. When the computer program is executed by a processor, the steps of the object processing method are implemented. Obtaining size information of each detection frame output by the detector; wherein each detection frame contains an object detected by the detector in the target image; Determining a size density state of all the detected objects in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the sizes of all the detected objects are uniform or non-uniform; Clustering all the detected objects in the target image based on a clustering algorithm associated with the size density state.

[0067] The specific embodiment process of the above method steps can be referred to Embodiment One, which will not be repeated here.

[0068] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0069] The above embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiment methods can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases, the former is a better embodiment.

[0071] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the present application specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An object processing method characterized by comprising: The method comprises the following steps: obtaining size information of each detection frame of the detector output; wherein each detection frame contains an object detected by the detector in the target image; determining the size density state of all detected objects in the target image based on the size information of each detection frame; wherein the size density state is used to represent whether the size of all detected objects is uniform or non-uniform; clustering all detected objects in the target image based on the clustering algorithm associated with the size density state.

2. The object processing method according to claim 1, characterized by, The step of determining the size density state of all detected objects in the target image based on the size information of each detection frame comprises: obtaining the height and width of all detection frames from all size information as a set of data, and calculating the variance of the set of data; when the variance is less than or equal to a preset variance threshold, determining that the size density state is used to represent that the size of all detected objects is uniform; when the variance is greater than the preset variance threshold, determining that the size density state is used to represent that the size of all detected objects is non-uniform.

3. The object processing method according to claim 1, characterized by, When the size density state is used to represent that the size of all detected objects is uniform, the step of clustering all detected objects in the target image based on the clustering algorithm associated with the size density state comprises: obtaining the height and width of all detection frames from all size information as a set of data, and calculating the average value of the set of data; obtaining a preset neighborhood radius weight, weighting the average value using the neighborhood radius weight to obtain an average weighted value; wherein the average weighted value is greater than the average value and less than or equal to twice the average value; using the average weighted value as the neighborhood radius in the DBSCAN algorithm, and calling the DBSCAN algorithm to cluster the detection frames of all detected objects.

4. The object processing method according to claim 1, characterized by, When the size density state is used to represent that the size of all detected objects is non-uniform, the step of clustering all detected objects in the target image based on the clustering algorithm associated with the size density state comprises: calculating the Euclidean distance between each two detection frames based on the center coordinates of the detection frames in all size information; calculating the size difference between each two detection frames based on the height and width of the detection frames in all size information; using the Euclidean distance and the size difference between each two detection frames as input parameters of a preset size perception distance formula to calculate the size perception distance between each two detection frames; constructing an N-dimensional matrix based on the size perception distance between each two detection frames; wherein N is the number of detection frames, and the size perception distance on the main diagonal line in the N-dimensional matrix is 0; using the N-dimensional matrix as the distance matrix in the OPTICS algorithm, and calling the OPTICS algorithm to cluster the detection frames of all detected objects.

5. The object processing method according to claim 4, characterized by, The step of using the Euclidean distance and the size difference between each two detection frames as input parameters of a preset size perception distance formula to calculate the size perception distance between each two detection frames comprises: obtaining a preset Euclidean distance weight and a size difference weight; wherein the Euclidean distance weight is less than the size difference weight. weight the Euclidean distance of each two bounding boxes by using the Euclidean distance weight, to obtain a Euclidean distance weighted value of each two bounding boxes; weight the size difference of each two bounding boxes by using the size difference weight, to obtain a size difference weighted value of each two bounding boxes; based on the Euclidean distance weighted value and the size difference weighted value of each two bounding boxes, calculate a size-aware distance of each two bounding boxes.

6. The object processing method according to claim 1, characterized by, After clustering all the detected objects in the target image based on the clustering algorithm associated with the size density state, the object processing method further comprises: filtering the size information of the bounding box of each cluster object from all the size information; obtaining all the corner point coordinates contained in the size information of the bounding box of each cluster object; performing convex hull calculation on all the corner point coordinates corresponding to each cluster object, to obtain convex hull vertex coordinates corresponding to each cluster object; based on the convex hull vertex coordinates corresponding to each cluster object, generating a semi-transparent mask for covering each cluster object.

7. The object processing method according to claim 6, wherein After generating the semi-transparent mask for covering each cluster object based on the convex hull vertex coordinates corresponding to each cluster object, the object processing method further comprises: in response to a mask selection instruction, determining a target semi-transparent mask selected by the mask selection instruction; calculating an overlap ratio of each bounding box and the target semi-transparent mask; wherein the overlap ratio is a ratio between an overlapping area of the bounding box and the target semi-transparent mask and an area of the bounding box; counting the number of bounding boxes whose overlap ratio is greater than or equal to a preset ratio threshold value; taking the counted number as the number of objects covered by the target semi-transparent mask.

8. An object processing apparatus characterized by comprising: comprises: an acquisition module, configured to acquire size information of each bounding box output by a detector; wherein each bounding box contains an object detected by the detector in a target image; a determination module, configured to determine a size density state of all the detected objects in the target image based on the size information of each bounding box; wherein the size density state is used to represent whether the sizes of all the detected objects are uniform or non-uniform; a clustering module, configured to cluster all the detected objects in the target image based on a clustering algorithm associated with the size density state.

9. A computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to execute the computer program to implement the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images