Material counting method , material positioning method, and apparatus

Abstract

Provided in the embodiments of the present application are a material counting method, a material positioning method, and an apparatus. A sample image comprises a plurality of materials and the areas where the materials are located are labeled, and the size of an area can reflect the size of a material to a certain extent. Therefore, size features are added in a training process, enabling a target material detection model to be compatible with materials of different sizes. Moreover, since the labeled areas do not overlap each other, the target material detection model can learn how to distinguish materials and gaps between the materials in the training process. Therefore, in an obtained confidence map, the materials have a higher level of confidence, while the gaps between the materials have a lower level of confidence, and thus each connected component formed by pixel points with pixel values greater than a confidence threshold value in the confidence map can be regarded as the area where one material is located, that is, the number of connected components is the number of materials. Therefore, the possibility of mistakenly recognizing a plurality of materials as one material can be effectively reduced, and thus the accuracy of counting is further improved.

Description

A material counting and positioning method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202410505015.7, filed on April 24, 2024, entitled "A Material Counting and Positioning Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of target counting and industrial vision technology, and in particular to a material counting and positioning method and apparatus. Background Technology

[0003] With the development of the electronics and information industry, more and more electronic products are emerging. All kinds of electronic products are controlled by circuit boards, most of which are composed of various electronic components of different numbers. During the production process, it is necessary to count and locate these electronic components. Current technology uses neural network models to count and locate electronic components by marking their center points on images. However, because only the center points are marked, this method has poor compatibility with materials of different sizes, resulting in low counting accuracy.

[0004] Therefore, there is an urgent need for a way to improve the accuracy of electronic component counting. Summary of the Invention

[0005] The purpose of this application is to provide a material counting and positioning method and apparatus to solve the problem of inaccurate counting of electronic components. The specific technical solution is as follows:

[0006] In a first aspect of this application, a material counting method is provided, the method comprising:

[0007] The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold.

[0008] The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0009] The quantity of material in the image to be detected is determined based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

[0010] In one possible implementation, the sample images are pre-annotated in the following ways:

[0011] Obtain a sample image with the material's center of gravity pixels marked;

[0012] For each material, a pixel region centered on the centroid pixel and marked with a size is determined in the sample image as a candidate material region, wherein the marked size is not greater than the expected size of the material;

[0013] From the candidate material areas, determine the target material area with the largest labeled size that does not overlap with other candidate material areas;

[0014] The target material area is labeled as the area where the material is located in the sample image.

[0015] In one possible implementation, the target material detection model is pre-trained in the following ways:

[0016] The sample image is input into the original material detection model to obtain a confidence map;

[0017] The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values.

[0018] The connected component of the first value is determined as the predicted region of the material in the sample image;

[0019] A loss function is constructed based on the difference between the predicted region and the region where the material is located as marked in the sample image, wherein the loss function is positively correlated with the difference;

[0020] The parameters of the original material detection model are adjusted in order to reduce the loss function, thereby obtaining the target material detection model.

[0021] In one possible implementation, determining the target material region with the largest labeled size that does not overlap with other candidate material regions from the candidate material regions includes:

[0022] If there is an overlapping area between the candidate material areas in the sample image, the annotation size is reduced by a preset step size, the reduced annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area with the annotation size centered on the centroid pixel in the sample image as the candidate material area.

[0023] If the candidate material areas of the sample image do not overlap, then the candidate material area with the largest labeled size is determined as the target material area, in the absence of overlapping areas; or,

[0024] If there is no overlapping area in the candidate material area of ​​the sample image, the annotation size is increased by a preset step size, the increased annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area in the sample image with the centroid pixel as the center and the annotation size as the candidate material area.

[0025] If there is an overlapping area among the candidate material areas in the sample image, then in the absence of an overlapping area, the candidate material area with the largest annotation size is determined as the target material area.

[0026] In one possible implementation, the existence of overlapping regions in the material areas of the sample image is determined by the following methods:

[0027] Determine the number of connected components in the candidate material region of the sample image;

[0028] If the number of centroid pixels of all materials in the sample image is greater than the number of connected components in the candidate material region, then there is an overlapping region in the candidate material region of the sample image.

[0029] If the number of centroid pixels of all materials in the sample image is equal to the number of connected regions in the candidate material region, then there is no overlapping region in the candidate material region of the sample image.

[0030] In one possible implementation, determining a pixel region centered on the centroid pixel and labeled with dimensions in the sample image as a candidate material region includes:

[0031] The shorter side of the boundary frame of the material is used as the dimension.

[0032] In the sample image, a circular pixel region centered on the centroid pixel and with the labeled size as the diameter is determined as the candidate material region.

[0033] In a second aspect of this application, a material positioning method is provided, the method comprising:

[0034] The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold.

[0035] The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0036] The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values.

[0037] The positions of the feature points in the connected domains of each of the first values ​​are determined as the positions of the materials.

[0038] In a third aspect of this application, a material counting device is provided, the device comprising:

[0039] The first acquisition module is used to acquire an image to be detected by taking a picture of a container holding multiple identical materials; wherein the size of the materials is smaller than a preset size threshold, and the number of materials is greater than a preset number threshold;

[0040] The first detection module is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0041] The first setting module is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values;

[0042] The statistics module is used to count the number of connected components of the first value, which is taken as the quantity of materials in the image to be detected.

[0043] In a fourth aspect of this application, a material positioning device is provided, the device comprising:

[0044] The second acquisition module acquires an image of a container holding multiple identical materials by taking a picture of the container; wherein the size of the materials is smaller than a preset size threshold and the number of materials is greater than a preset number threshold.

[0045] The second detection module is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0046] The second setting module is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values;

[0047] The material determination module is used to determine the position of the feature point in the connected domain of each of the first values, as the position of the material.

[0048] In a fifth aspect of this application, an electronic device is provided, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor, when executing a program stored in memory, implements the method described in either the first or second aspect above.

[0051] Beneficial effects of the embodiments in this application:

[0052] This application provides a material location and counting method and apparatus. It acquires a target image by photographing a container holding multiple identical materials, inputs the target image into a pre-trained target material detection model, and obtains a confidence map. The quantity of materials in the target image is determined based on the number of connected components in the confidence map where the pixel value is greater than a confidence threshold. In this application embodiment, the target material detection model uses multiple materials and labels the areas where the materials are located. That is, the sample image includes multiple materials and labels the areas where the materials are located. The size of the area can reflect the size of the materials to a certain extent; therefore, size features are incorporated during training, enabling the target material detection model to be compatible with materials of different sizes. Meanwhile, since the labeled areas do not overlap, the target material detection model can learn how to distinguish materials and gaps between them during training. As a result, the confidence level of materials in the confidence map is relatively high, while the confidence level of gaps between materials is relatively low. Therefore, each connected region formed by pixels with pixel values ​​greater than the confidence threshold in the confidence map can be regarded as the region where a material is located. In other words, the number of connected regions is the number of materials. This can effectively reduce the possibility of misidentifying multiple materials as one material, thereby improving the accuracy of counting.

[0053] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. Attached Figure Description

[0054] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0055] Figure 1-1 is a schematic diagram of a material technology method flow provided in an embodiment of this application;

[0056] Figure 1-2a is a schematic diagram of a material provided in an embodiment of this application;

[0057] Figure 1-2b is a schematic diagram of another material provided in an embodiment of this application;

[0058] Figure 2-1 is a schematic diagram of a sample image annotation method provided in an embodiment of this application;

[0059] Figure 2-2a is a schematic diagram of an electronic component provided in an embodiment of this application;

[0060] Figure 2-2b is a schematic diagram of another electronic component provided in an embodiment of this application;

[0061] Figures 2-3 are schematic diagrams of sample images marked with centroid pixels and background provided in an embodiment of this application;

[0062] Figure 2-4a is a schematic diagram of the expected dimensions of a material provided in an embodiment of this application;

[0063] Figure 2-4b is a schematic diagram of the expected dimensions of another material provided in an embodiment of this application;

[0064] Figures 2-5 are schematic diagrams of a circular pixel area centered on the centroid pixel and labeled with a diameter, provided in an embodiment of this application.

[0065] Figures 2-6 are schematic diagrams illustrating the overlapping of circles and squares of the same size and position according to an embodiment of this application;

[0066] Figure 3-1a is a schematic diagram of determining the position of the centroid pixel of a predicted material according to an embodiment of this application;

[0067] Figure 3-1b is a schematic diagram of another method for determining the position of the centroid pixel of the predicted material according to an embodiment of this application;

[0068] Figure 3-2a is a schematic diagram of a counting result provided in an embodiment of this application;

[0069] Figure 3-2b is a schematic diagram of another counting result provided by an embodiment of this application;

[0070] Figure 4a is a schematic diagram of different annotation sizes provided in an embodiment of this application;

[0071] Figure 4b is a schematic diagram of another different dimension annotation provided in an embodiment of this application;

[0072] Figure 5 is a schematic flowchart of a material positioning method provided in an embodiment of this application;

[0073] Figure 6 is a schematic diagram of a material counting device provided in an embodiment of this application;

[0074] Figure 7 is a schematic diagram of a material positioning device provided in an embodiment of this application;

[0075] Figure 8 is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention are within the scope of protection of this invention.

[0077] This application provides a material counting method, as shown in Figure 1-1, which includes:

[0078] S101. Acquire the image to be detected by taking a picture of a container holding multiple identical materials.

[0079] S102. Input the image to be detected into the pre-trained target material detection model to obtain a confidence map.

[0080] S103. Determine the quantity of materials in the image to be detected based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

[0081] In this embodiment, on the one hand, since the target material detection model is trained using sample images of multiple materials, and the sample images are labeled with the regions where the materials are located, meaning the sample images include multiple materials and label the regions where the materials are located, and the size of the regions can reflect the size of the materials to a certain extent, size features are added during the training process, enabling the target material detection model to be compatible with materials of different sizes. Simultaneously, since the labeled regions do not overlap, the target material detection model can learn how to distinguish between materials and the gaps between them during training. Therefore, the confidence level of materials in the resulting confidence map is relatively high, while the confidence level of the gaps between materials is relatively low. Thus, each connected component formed by pixels with pixel values ​​greater than the confidence threshold in the confidence map can be considered as the region where a material is located; that is, the number of connected components is the number of materials. This effectively reduces the possibility of misidentifying multiple materials as a single material, thereby improving the accuracy of counting.

[0082] On the other hand, in this embodiment, when marking the area where the material is located, it is only necessary to ensure that the marked areas do not overlap, without needing to accurately mark the outer bounding box of the material, which can effectively reduce marking costs. In this application, the outer bounding box of the material refers to the outer contour of the material.

[0083] The following will describe S101-S103 in detail:

[0084] In S101, the material size is smaller than a preset size threshold, and the material quantity is greater than a preset quantity threshold. The preset size threshold and preset quantity threshold in this document can be set according to the user's actual needs or experience, and this application does not impose any restrictions on this. However, the preset size threshold should be sufficiently small, and the preset quantity threshold should be sufficiently large, meaning that the quantity of the material can be considered difficult to obtain directly through manual counting.

[0085] The materials in this embodiment are shown in Figure 1-2a. The application scenario of this application is suitable for small-sized materials in large quantities. In the image to be detected, the number of materials is large, making accurate and rapid manual counting impossible. Figure 1-2a shows a material image in a real-world scenario, illustrating the actual application scenario of this application. However, Figure 1-2a does not clearly show the size, shape, and other information of the materials. To more clearly show the size, shape, and other information of the materials, as shown in Figure 1-2b, this embodiment also provides a material schematic diagram corresponding to Figure 1-2a.

[0086] In S102, the pixels in the confidence map correspond to the pixels in the image to be detected. The pixel value of each pixel in the confidence map represents the probability that the corresponding pixel in the image to be detected belongs to a material region. The target material detection model is trained using sample images. The sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap. The pixel value of each pixel in the confidence map can be positively or negatively correlated with the probability that the labeled pixel belongs to a material. For ease of description, the following only uses the positive correlation as an example. For instance, the pixel value of each pixel in the confidence map represents the probability that the pixel belongs to a material. The pixel value ranges from 0 to 255; the closer the pixel value is to 255, the higher the probability that the pixel belongs to a material.

[0087] In S103, since the material is a complete region, the connected component of a pixel in the confidence map is very likely to be a material. Therefore, when determining the number of materials in the image to be detected, it is based on the number of connected components of pixels in the confidence map whose pixel values ​​are greater than the confidence threshold.

[0088] To further improve the accuracy of electronic component counting, the sample images in this application are pre-annotated in the following manner, as shown in Figure 2-1. This method includes:

[0089] S201. Obtain a sample image with the material's center of gravity pixel marked.

[0090] S202. For each material, determine a pixel area in the sample image centered on the centroid pixel and marked with dimensions, as a candidate material area. The marked dimensions shall not be larger than the expected size of the material.

[0091] S203. Determine the target material area from the candidate material areas that has the largest labeled size and does not overlap with other candidate material areas.

[0092] S204. Mark the target material area as the area where the material is located in the sample image.

[0093] In this embodiment, the target material region is used as the ground truth value of the region where the material is located in the sample image for training. This increases the training of material features in the target material detection model, thereby improving the accuracy of the target material detection model. Since the added material features are related to the size of the material, the target material detection model can be compatible with materials of different sizes. In addition, the material region is determined based on the centroid pixel of the material. The centroid pixel can represent the distribution of all pixels within the material region. Since the pixel distribution in the material region is uniform, the centroid pixel can be approximated as the center of the material region. In this way, the material region determined by the centroid pixel and the labeled size can comprehensively represent the characteristics of the material, and the determined target region is a non-overlapping area. This can increase the material features while ensuring that no material region is missed, thus improving the accuracy of material counting.

[0094] The above S201-S204 will be described in detail below.

[0095] In S201, after acquiring the sample image, all materials in the sample image are labeled. The labeling method can be to mark the centroid pixels of the materials. These centroid pixels are used to represent material labels, indicating that the area contains the material. When a centroid pixel is marked, it indicates that material exists within the connected domain where that centroid pixel is located. In this embodiment, the materials are electronic components on an electronic device, such as capacitors, resistors, and inductors. As shown in Figure 2-2a, each small box represents an integrated circuit (IC) component. To more clearly illustrate the size, shape, and other information of the electronic components, as shown in Figure 2-2b, this application embodiment also provides a schematic diagram of an electronic component corresponding to Figure 2-2a, where each small box represents an integrated circuit component.

[0096] In this embodiment, the material type and size are the same in each sample image. However, the material type and size in different sample images can be the same or different. To better train the target material detection model, the material type and size in different sample images should be as different as possible. This results in different material features, increasing the variety of material features during training and thus improving the compatibility of the trained target material detection model. To better distinguish the material from the background area, the pixel value of the centroid pixel can be set to a first value, and the pixel values ​​of all other pixels can be set to a second value. The range of the first and second values ​​is 0-255. To more clearly distinguish the centroid pixel from other pixels, the difference between the first and second values ​​should be as large as possible. For example, the first value can be set to 255 and the second value to 0; or the first value can be set to 193 and the second value to 62. For example, by using binarization to process the sample image, the centroid pixels in the sample image can be distinguished from other pixels, as shown in Figure 2-3. Each white dot is the centroid pixel of the material, and the black area is the background area. The black background area also includes non-centroid pixels of the material, that is, there will be material areas around the white dots. In the future, some non-centroid pixels of the material will be identified as material areas. How to determine the material areas will be described in detail later, and will not be elaborated on here.

[0097] In S202, for each material, a candidate material region is determined, which is the area where the material is located. The reason for determining a material region is to incorporate the material's dimensions into the model training process.

[0098] For ease of calculation, the expected size in this embodiment is the material size determined according to the actual outline of the material. This material size is not the actual outline of the material, but a relatively regular outline. For example, the material may be irregularly shaped, and the expected size is the length, width, or height of the regular shape circumscribed by the smallest area of ​​the material, etc., which needs to be determined according to the actual application scenario. As shown in Figure 2-4a, the length and width of the rectangular area surrounding the material are the expected size of the material. To more clearly illustrate the size, shape, and other information of the material, as shown in Figure 2-4b, this embodiment also provides a schematic diagram of the expected size of a material corresponding to Figure 2-4a. The length and width of the outermost rectangle in Figure 2-4b are the expected size of the material, and the lines inside the rectangle are used to represent the shape of the material.

[0099] The dimension used to label the material area when determining the candidate material area should not be larger than the expected size. In this embodiment, when labeling the area where the material is located, it is only necessary to ensure that the labeled areas do not overlap, without needing to accurately label the outer bounding box of the material, which can effectively reduce labeling costs.

[0100] In one possible embodiment, in order to make the size of the candidate material area more closely match the actual area of ​​the material, thereby improving the accuracy of the subsequent target material detection model, S202 specifically includes:

[0101] S2021. Use the shorter side of the material's bounding box as the dimension.

[0102] In this step, the bounding box of the material is a contour box determined based on the pixels at the material's boundary location. The boundary location of the material is where it meets the background.

[0103] In this embodiment, all areas except the material area are background areas. The shorter side of the bounding box is the shorter side within the bounding box. Since the bounding box is rectangular, the shorter side of the rectangle's length and width is selected as the shorter side. In another possible embodiment, the bounding box can also be a pentagon, in which case twice the shortest distance from the center of the pentagon to each side is used as the shorter side. The specific method for determining the shorter side of the material's bounding box is determined by those skilled in the art based on the shape of the bounding box in the actual application scenario. Because a rectangle can completely enclose various shapes, and a rectangular bounding box is easier to calculate, most material bounding boxes are rectangular.

[0104] S1022. In the sample image, determine a circular pixel area with the centroid pixel as the center and the labeled size as the diameter, as the candidate material area.

[0105] In this step, as shown in Figure 2-5, the white area represents the pixel region containing a circle centered at the centroid pixel with a diameter of d. For ease of understanding, the center is represented by a black dot. When labeling the material area, to further reduce overlapping areas, the material area can be defined as a circle. This is because, compared to other shapes of the same size, circles have a lower probability of overlapping. For example, as shown in Figure 2-6, a circle with diameter d and a square with side length d are in the same position as two squares; the two circles do not overlap, but the two squares do. Therefore, it can be concluded that since the area of ​​a square is larger than that of a circle of the same size, the probability of squares overlapping is greater than that of circles. Therefore, using the pixel region containing a circle centered at the centroid pixel with a diameter of d as the material area can reduce the probability of material overlap to a certain extent.

[0106] In S203, to ensure no subsequent omissions in detection, the target material area must first not overlap with other candidate material areas. Secondly, since this embodiment requires adding material features, the target material area needs to have the largest possible annotation size to better characterize the material. Thus, the annotation sizes for materials of different sizes will differ. The candidate material area with the largest annotation size best represents the area where the material is located. If the annotation size is larger than the largest annotation size, the candidate material areas will overlap; if the annotation size is smaller than the largest annotation size, some material areas will not be within this area.

[0107] In S204, the target material region is the region where the material is located in the sample image. In other words, the material features are added to the sample image, and these features are related to the size of the material region. Therefore, the trained target material detection model can be compatible with materials of different sizes, which can improve the accuracy of the target material detection model.

[0108] In this application embodiment, the target material detection model is pre-trained in the following ways, including:

[0109] A. Input the sample image into the original material detection model to obtain the confidence map.

[0110] B. Set the pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold to the first value, and set the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold to other values.

[0111] In this step, pixels with a confidence threshold greater than the threshold are highly likely to be material pixels, while pixels with a confidence threshold less than the threshold are highly unlikely to be material pixels. In other words, the first value is highly likely to be a material pixel, and other values ​​are highly unlikely to be material pixels. Essentially, material pixels and non-material pixels are distinguished using two different pixel values ​​to more clearly show the difference between materials and non-material pixels in the confidence map.

[0112] C. Determine the connected region of the first value as the prediction region of the material in the sample image.

[0113] D. Construct a loss function based on the difference between the predicted region and the region where the material is located as marked in the sample image.

[0114] The loss function is positively correlated with the difference.

[0115] E. Adjust the parameters of the original material detection model in the direction of reducing the loss function to obtain the target material detection model.

[0116] When determining the target material area, it is necessary to identify a target material area that does not overlap with other candidate material areas. This application provides a method for determining the target material area. Since the target material area must meet two conditions: it must not overlap with other candidate material areas; and its dimension must be the largest possible, this application provides two approaches to determine the target material area:

[0117] The first method involves initially setting the dimension to a large value, then continuously reducing the dimension until the resulting candidate material area meets the above conditions. This candidate material area is then designated as the target material area. Specifically, this includes:

[0118] a. If there is an overlapping area between the candidate material areas in the sample image, the annotation size is reduced by a preset step size, the reduced annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area with the centroid pixel as the center and the annotation size in the sample image as the candidate material area.

[0119] b. If there is no overlapping area between the candidate material areas in the sample image, then the candidate material area with the largest annotation size is determined as the target material area in the absence of overlapping areas.

[0120] In this embodiment, if the candidate material areas of the sample image have overlapping regions, it indicates that the initial annotation size of the candidate material area is too large, resulting in multiple centroid pixels being located within a single connected component. This means multiple materials are considered as one material, making them indistinguishable. Therefore, to better differentiate the materials and divide them into the same number of connected components, the annotation size of the candidate material area needs to be adjusted. This requires reducing the annotation size and then redefining the candidate material area.

[0121] If there are no overlapping areas in the candidate material areas of the sample image, it means that the annotation size of the candidate material area is appropriate and can distinguish each material. Therefore, the candidate material area is taken as the target material area.

[0122] The second method involves initially setting the dimension to a small value and then continuously increasing it until the resulting candidate material area meets the above conditions. This candidate material area is then designated as the target material area. Specifically, this includes:

[0123] c. If there is no overlapping area in the candidate material area of ​​the sample image, the annotation size is increased according to the preset step size, the increased annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area with the centroid pixel as the center and the annotation size as the candidate material area in the sample image.

[0124] d. If there is an overlapping area between the candidate material areas in the sample image, then the candidate material area with the largest annotation size is determined as the target material area if there is no overlapping area.

[0125] In this embodiment, if there is no overlapping area in the candidate material area of ​​the sample image, the candidate material area is well separated because the initial setting of the annotation size is a small value. However, in order to meet the maximum annotation size, the annotation size needs to be adjusted slowly until there is an overlapping area in the candidate material area of ​​the sample image. Once there is an overlapping area, it means that the annotation size of the candidate material area has reached the critical value. Increasing it a little more will cause overlap. Therefore, the annotation size closest to the critical value is the maximum annotation size.

[0126] In this embodiment of the application, the existence of overlapping regions in the material areas of a sample image is determined by the following method:

[0127] I) Determine the number of connected components in the candidate material region of the sample image.

[0128] In this step, the number of connected components in the candidate material region is the number of connected components formed by the candidate material region in the sample image, excluding the connected components of the background region.

[0129] II) If the number of centroid pixels of all materials in the sample image is greater than the number of connected regions in the candidate material region, then there is an overlapping region in the candidate material region of the sample image.

[0130] III) If the number of centroid pixels of all materials in the sample image is equal to the number of connected regions in the candidate material region, then there is no overlapping region in the candidate material region of the sample image.

[0131] In the above steps, since each material has a centroid pixel, the total number of centroid pixels in the sample image is equal to the total number of materials in the sample image. If the total number of centroid pixels in the sample image is less than the number of connected components in the candidate material regions, it means that the candidate material regions containing materials have been merged into a single connected component. If the total number of centroid pixels in the sample image is equal to the number of connected components in the candidate material regions, it means that the candidate material regions without materials have been merged into a single connected component.

[0132] In this embodiment, since if the materials do not overlap, the connected components corresponding to each candidate material region should be independent, and the number of connected components should be the same as the number of materials in the sample image, the number of connected components in the candidate material regions in the sample image is determined to determine whether there is an overlapping region in the candidate regions of the sample image. This eliminates the need to use complex algorithms to determine whether there is an overlapping region, thereby improving the efficiency of determining whether there is an overlapping region in the candidate regions of the sample image.

[0133] After training the target material detection model, in one possible embodiment, based on the target detection model trained by the aforementioned model training method, this application also provides a material counting method:

[0134] S301. Acquire the image to be detected by taking a picture of a container holding multiple identical materials.

[0135] This step is the same as S101 above, and will not be described in detail here.

[0136] S302. Input the image to be detected into the pre-trained target material detection model to obtain a confidence map.

[0137] This step is the same as S102 mentioned above, and will not be described in detail here.

[0138] S303. Determine the quantity of materials in the image to be detected based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

[0139] In this step, the pixel location with the maximum value in a local area of ​​the confidence map is determined as the centroid pixel location of the predicted material. As shown in Figure 3-1a, within a certain white circular area, if the pixel value of the center pixel is greater than the pixel values ​​of all surrounding pixels, this is the maximum value, and this center location is selected as the predicted material location within that local area. In one possible embodiment, the confidence map can also be used to locate or count the materials in the target image, depending on the specific application scenario.

[0140] Figure 3-1a distinguishes the central pixel from the surrounding pixels within a white circular area by color. The surrounding pixels within this white circular area are gray circles, while the central pixel is a black circle in the center. The colors of the surrounding pixels are not very distinct. To more clearly show the central pixel from the surrounding pixels, as shown in Figure 3-1b, this embodiment also provides a schematic diagram corresponding to Figure 3-1a for determining the position of the centroid pixel of a predicted material. In Figure 3-1b, the black-filled circle represents the central pixel within the white circular area, corresponding to the black circle in Figure 3-1a, and the circle filled with "+" represents the surrounding pixels within the white circular area, corresponding to the gray circle in Figure 3-1a.

[0141] The count result is the number of predicted centroid pixels of the material, and the position of the predicted centroid pixel is the location of the material. Figure 3-2a shows a schematic diagram of the count result, where the number 1501 at the center of the disk represents the total number of materials enclosed by the boxes as 1501. To more clearly show the size, shape, and other information of the materials, as shown in Figure 3-2b, this embodiment also provides a schematic diagram of the count result corresponding to Figure 3-2a. In Figure 3-2b, the number 1501 at the center of the disk represents the total number of materials enclosed by the boxes as 1501. The number 1501 at the center of the disk in Figures 1-2a and 1-2b also represents the total number of materials enclosed by the boxes as 1501.

[0142] In this embodiment, the confidence map obtained by the target material detection model is used to count the materials in the image to be detected. Since the size of the materials is introduced during the training process of the target material detection model, the target material detection model can be compatible with materials of different sizes, thereby improving the accuracy of the target material detection model in counting.

[0143] After obtaining the image to be detected, the specific steps for material counting include:

[0144] S401. Obtain the bounding box of each material in the image to be detected, which is marked with the centroid pixels of the material.

[0145] This step is the same as the process of obtaining the bounding box size in S202 above, and will not be described in detail here.

[0146] S402. For each material, use the shorter side of the material's bounding box as the dimension.

[0147] This step is the same as S2021 above, and will not be elaborated further here. Figure 4a shows a schematic diagram of different labeling sizes. Larger components have larger labeling sizes, while smaller components have smaller labeling sizes; that is, large-sized materials are labeled with large sizes, and small-sized materials with small sizes. To more clearly show the size, shape, and other information of the materials, as shown in Figure 4b, this embodiment also provides a schematic diagram of different labeling sizes corresponding to Figure 4a. In Figure 4b, larger components have larger labeling sizes, while smaller components have smaller labeling sizes; that is, large-sized materials are labeled with large sizes, and small-sized materials with small sizes.

[0148] S403. In the sample image, determine a circular pixel area centered on the centroid pixel and labeled with a diameter as the area to be selected as the candidate material area.

[0149] This step is the same as S2022 mentioned above, and will not be elaborated on further here.

[0150] S404. Determine the number of connected components in the candidate material region of the sample image.

[0151] In this step, the number of connected components in the candidate material region is the number of connected components in the sample image where the candidate material region is located, excluding the connected components of the background region.

[0152] S405. If the number of centroid pixels of all materials in the sample image is equal to the number of connected regions in the candidate material region, then there is no overlapping area in the candidate material region of the sample image. Increase the annotation size according to the preset step size and return to execute S402.

[0153] S406. If the number of centroid pixels of all materials in the sample image is greater than the number of connected regions in the candidate material region, then there is an overlapping area in the candidate material region of the sample image. In the case that there is no overlapping area, the candidate material region with the largest annotation size is determined as the target material region.

[0154] The steps S405-S406 described above are the same as the steps for determining the target material area, and will not be elaborated further here.

[0155] S407. Train the model using the target material region as the ground truth of the material region in the sample image to obtain the target material detection model used to determine the material region in the image.

[0156] This step is the same as S204 above, and will not be elaborated on further here.

[0157] S408. Input the image to be detected into the target material detection model to obtain a confidence map; wherein, the pixel value of the pixel in the confidence map is used to represent the probability that the pixel belongs to the material.

[0158] This step is the same as S102 mentioned above, and will not be described in detail here.

[0159] S409. Determine the quantity of materials in the image to be detected based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

[0160] This step is the same as S103 mentioned above, and will not be described in detail here.

[0161] The above steps S401-S406 describe the training process of the target material detection model, while S408-S409 describe the counting process using the target material detection model. The target material detection model is not retrained every time a counting is performed; it is pre-trained and only needs to be trained once.

[0162] In this embodiment, the target material region is used as the ground truth value of the region where the material is located in the sample image for training. This increases the training of material features in the target material detection model, thereby improving the accuracy of the target material detection model. Since the added material features are related to the size of the material, the target material detection model can be compatible with materials of different sizes. In addition, the material region is determined based on the centroid pixel of the material. The centroid pixel can represent the distribution of all pixels within the material region. Since the pixel distribution in the material region is uniform, the centroid pixel can be approximated as the center of the material region. In this way, the material region determined by the centroid pixel and the labeled size can comprehensively represent the characteristics of the material. The determined target region is a non-overlapping area, which ensures that no material region is missed while increasing the material features, thus improving the accuracy of material counting.

[0163] This application embodiment also provides a material positioning method, as shown in Figure 5, the method includes:

[0164] S501. Acquire an image of the object to be detected by photographing a container holding multiple identical materials.

[0165] This step is the same as S101 above, and will not be described in detail here.

[0166] S502. Input the image to be detected into the pre-trained target material detection model to obtain a confidence map.

[0167] This step is the same as S101 above, and will not be described in detail here.

[0168] S503. Set the pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold to the first value; and set the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold to other values.

[0169] S504. Determine the position of the feature point in the connected domain of each first value, and use it as the position of the material.

[0170] In this embodiment, on the one hand, since the labeled areas do not overlap, the target material detection model can learn how to distinguish materials and gaps between them during training. Therefore, the confidence level of the materials in the confidence map is relatively high, and the confidence level is higher closer to the center of the material, while the confidence level of the gaps between materials is relatively low. Therefore, the pixel values ​​in the confidence map should gradually decrease from the center of the material outwards. Thus, the center of the connected region formed by pixels with pixel values ​​greater than the confidence threshold can be regarded as the center of the material, thereby realizing the localization of the material. At the same time, since the target material detection model is trained on sample images of multiple materials and the areas where the materials are located are labeled in the sample images, that is, the sample images include multiple materials and the areas where the materials are located are labeled. The size of the area can reflect the size of the material to a certain extent. Therefore, size features are added during training, so that the target material detection model can be compatible with materials of different sizes, thereby improving the accuracy of localization.

[0171] On the other hand, in the embodiments of this application, when marking the area where the material is located, it is only necessary to ensure that the marked areas do not overlap, without needing to accurately mark the outer bounding box of the material, which can effectively reduce the marking cost.

[0172] This application embodiment also provides a material counting device, as shown in FIG6, the device comprising:

[0173] The first acquisition module 601 is used to acquire an image to be detected by taking a picture of a container holding multiple identical materials; wherein the size of the materials is smaller than a preset size threshold and the number of materials is greater than a preset number threshold.

[0174] The first detection module 602 is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0175] The first setting module 603 is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values;

[0176] The statistics module 604 is used to count the number of connected components of the first value, which is used as the quantity of materials in the image to be detected.

[0177] In this embodiment, on the one hand, since the target material detection model is trained using sample images of multiple materials, and the sample images are labeled with the regions where the materials are located, meaning the sample images include multiple materials and label the regions where the materials are located, and the size of the regions can reflect the size of the materials to a certain extent, size features are added during the training process, enabling the target material detection model to be compatible with materials of different sizes. Simultaneously, since the labeled regions do not overlap, the target material detection model can learn how to distinguish between materials and the gaps between them during training. Therefore, the confidence level of materials in the resulting confidence map is relatively high, while the confidence level of the gaps between materials is relatively low. Thus, each connected component formed by pixels with pixel values ​​greater than the confidence threshold in the confidence map can be considered as the region where a material is located; that is, the number of connected components is the number of materials. This effectively reduces the possibility of misidentifying multiple materials as a single material, thereby improving the accuracy of counting.

[0178] On the other hand, in the embodiments of this application, when marking the area where the material is located, it is only necessary to ensure that the marked areas do not overlap, without needing to accurately mark the outer bounding box of the material, which can effectively reduce the marking cost.

[0179] This application embodiment also provides a material positioning device, as shown in FIG7, the device comprising:

[0180] The second acquisition module 701 acquires an image to be detected by photographing a container holding multiple identical materials; wherein the size of the materials is smaller than a preset size threshold, and the number of materials is greater than a preset number threshold.

[0181] The second detection module 702 is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0182] The second setting module 703 is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values;

[0183] The material determination module 704 is used to determine the position of the feature point in the connected domain of each of the first values, as the position of the material.

[0184] In this embodiment, on the one hand, since the labeled areas do not overlap, the target material detection model can learn how to distinguish materials and gaps between them during training. Therefore, the confidence level of the materials in the confidence map is relatively high, and the confidence level is higher closer to the center of the material, while the confidence level of the gaps between materials is relatively low. Therefore, the pixel values ​​in the confidence map should gradually decrease from the center of the material outwards. Thus, the center of the connected region formed by pixels with pixel values ​​greater than the confidence threshold can be regarded as the center of the material, thereby realizing the localization of the material. At the same time, since the target material detection model is trained on sample images of multiple materials and the areas where the materials are located are labeled in the sample images, that is, the sample images include multiple materials and the areas where the materials are located are labeled. The size of the area can reflect the size of the material to a certain extent. Therefore, size features are added during training, so that the target material detection model can be compatible with materials of different sizes, thereby improving the accuracy of localization.

[0185] On the other hand, in the embodiments of this application, when marking the area where the material is located, it is only necessary to ensure that the marked areas do not overlap, without needing to accurately mark the outer bounding box of the material, which can effectively reduce the marking cost.

[0186] This application also provides an electronic device, as shown in FIG8, including:

[0187] Memory 801 is used to store computer programs;

[0188] When processor 802 executes a program stored in memory 801, it performs the following steps:

[0189] The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold.

[0190] The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0191] The quantity of material in the image to be detected is determined based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

[0192] or,

[0193] The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold.

[0194] The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap;

[0195] The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values.

[0196] The positions of the feature points in the connected domains of each of the first values ​​are determined as the positions of the materials.

[0197] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 802, the communication interface, and the memory 801 communicating with each other via the communication bus.

[0198] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0199] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0200] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0201] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0202] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described model training methods.

[0203] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the model training methods described in the above embodiments.

[0204] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0205] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0206] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0207] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A material counting method, characterized in that, The method includes: The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold. The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The quantity of material in the image to be detected is determined based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence graph.

2. The method according to claim 1, characterized in that, The sample images are pre-annotated in the following ways: Obtain a sample image with the material's center of gravity pixels marked; For each material, a pixel region centered on the centroid pixel and marked with a size is determined in the sample image as a candidate material region, wherein the marked size is not greater than the expected size of the material; From the candidate material areas, determine the target material area with the largest labeled size that does not overlap with other candidate material areas; The target material area is labeled as the area where the material is located in the sample image.

3. The method according to claim 2, characterized in that, The target material detection model is pre-trained using the following methods: The sample image is input into the original material detection model to obtain a confidence map; The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values. The connected component of the first value is determined as the predicted region of the material in the sample image; A loss function is constructed based on the difference between the predicted region and the region where the material is located as marked in the sample image, wherein the loss function is positively correlated with the difference; The parameters of the original material detection model are adjusted in order to reduce the loss function, thereby obtaining the target material detection model.

4. The method according to claim 2, characterized in that, The step of determining the target material region with the largest labeled size that does not overlap with other candidate material regions from the candidate material regions includes: If there is an overlapping area between the candidate material areas in the sample image, the annotation size is reduced by a preset step size, the reduced annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area with the annotation size centered on the centroid pixel in the sample image as the candidate material area. If the candidate material areas of the sample image do not overlap, then the candidate material area with the largest labeled size is determined as the target material area, in the absence of overlapping areas; or, If there is no overlapping area in the candidate material area of ​​the sample image, the annotation size is increased by a preset step size, the increased annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area in the sample image with the centroid pixel as the center and the annotation size as the candidate material area. If there is an overlapping area among the candidate material areas in the sample image, then in the absence of an overlapping area, the candidate material area with the largest annotation size is determined as the target material area.

5. The method according to claim 2, characterized in that, The following methods are used to determine whether there are overlapping areas in the material regions of the sample image: Determine the number of connected components in the candidate material region of the sample image; If the number of centroid pixels of all materials in the sample image is greater than the number of connected components in the candidate material region, then there is an overlapping region in the candidate material region of the sample image. If the number of centroid pixels of all materials in the sample image is equal to the number of connected regions in the candidate material region, then there is no overlapping region in the candidate material region of the sample image.

6. The method according to claim 2, characterized in that, The step of determining a pixel region centered on the centroid pixel and labeled with dimensions in the sample image as a candidate material region includes: The shorter side of the boundary frame of the material is used as the dimension. In the sample image, a circular pixel region centered on the centroid pixel and with the labeled size as the diameter is determined as the candidate material region.

7. A material positioning method, characterized in that, The method includes: The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold. The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values. The positions of the feature points in the connected domains of each of the first values ​​are determined as the positions of the materials.

8. A material counting device, characterized in that, The device includes: The first acquisition module is used to acquire an image to be detected by taking a picture of a container holding multiple identical materials; wherein the size of the materials is smaller than a preset size threshold, and the number of materials is greater than a preset number threshold; The first detection module is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The first setting module is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values; The statistics module is used to count the number of connected components of the first value, which is taken as the quantity of materials in the image to be detected.

9. A material positioning device, characterized in that, The device includes: The second acquisition module acquires an image of a container holding multiple identical materials by taking a picture of the container; wherein the size of the materials is smaller than a preset size threshold and the number of materials is greater than a preset number threshold. The second detection module is used to input the image to be detected into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The second setting module is used to set the pixel values ​​of pixels in the confidence image that are greater than the confidence threshold to a first value; and to set the pixel values ​​of pixels in the confidence image that are less than the confidence threshold to other values; The material determination module is used to determine the position of the feature point in the connected domain of each of the first values, as the position of the material.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; When a processor executes a program stored in memory, it performs the following steps: The method involves acquiring an image of a container holding multiple identical materials by photographing the container itself; wherein the material size is smaller than a preset size threshold, and the number of materials is greater than a preset quantity threshold. The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The quantity of material in the image to be detected is determined based on the number of connected components of pixels whose pixel values ​​are greater than the confidence threshold in the confidence map. Alternatively, an image of the container containing multiple identical materials can be acquired by photographing the container itself; wherein the material size is smaller than a preset size threshold and the material quantity is greater than a preset quantity threshold. The image to be detected is input into a pre-trained target material detection model to obtain a confidence map; wherein, the pixels in the confidence map correspond to the pixels in the image to be detected, and the pixel value of the pixel in the confidence map is used to represent the probability that the corresponding pixel in the image to be detected belongs to the material region; the target material detection model is trained through sample images; the sample images include multiple materials and are labeled with the regions where the materials are located, and the labeled regions do not overlap; The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values. The positions of the feature points in the connected domains of each of the first values ​​are determined as the positions of the materials.

11. The electronic device according to claim 10, characterized in that, The processor is also used to perform the following steps: Obtain a sample image with the material's center of gravity pixels marked; For each material, a pixel region centered on the centroid pixel and marked with a size is determined in the sample image as a candidate material region, wherein the marked size is not greater than the expected size of the material; From the candidate material areas, determine the target material area with the largest labeled size that does not overlap with other candidate material areas; The target material area is labeled as the area where the material is located in the sample image.

12. The electronic device according to claim 11, characterized in that, The processor is also used to perform the following steps: The sample image is input into the original material detection model to obtain a confidence map; The pixel values ​​of pixels in the confidence graph that are greater than the confidence threshold are set to a first value; and the pixel values ​​of pixels in the confidence graph that are less than the confidence threshold are set to other values. The connected component of the first value is determined as the predicted region of the material in the sample image; A loss function is constructed based on the difference between the predicted region and the region where the material is located as marked in the sample image, wherein the loss function is positively correlated with the difference; The parameters of the original material detection model are adjusted in order to reduce the loss function, thereby obtaining the target material detection model.

13. The electronic device according to claim 11, characterized in that, The processor is also used to perform the following steps: If there is an overlapping area between the candidate material areas in the sample image, the annotation size is reduced by a preset step size, the reduced annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area with the annotation size centered on the centroid pixel in the sample image as the candidate material area. If the candidate material areas of the sample image do not overlap, then the candidate material area with the largest labeled size is determined as the target material area, in the absence of overlapping areas; or, If there is no overlapping area in the candidate material area of ​​the sample image, the annotation size is increased by a preset step size, the increased annotation size is used as the new annotation size, and the process returns to the step of determining the pixel area in the sample image with the centroid pixel as the center and the annotation size as the candidate material area. If there is an overlapping area among the candidate material areas in the sample image, then in the absence of an overlapping area, the candidate material area with the largest annotation size is determined as the target material area.

14. The electronic device according to claim 11, characterized in that, The processor is also used to perform the following steps: Determine the number of connected components in the candidate material region of the sample image; If the number of centroid pixels of all materials in the sample image is greater than the number of connected components in the candidate material region, then there is an overlapping region in the candidate material region of the sample image. If the number of centroid pixels of all materials in the sample image is equal to the number of connected regions in the candidate material region, then there is no overlapping region in the candidate material region of the sample image.

15. The electronic device according to claim 11, characterized in that, The processor is also used to perform the following steps: The shorter side of the boundary frame of the material is used as the dimension. In the sample image, a circular pixel region centered on the centroid pixel and with the labeled size as the diameter is determined as the candidate material region.

Images