Labelled random seed training sample generation method, detection method and device

Abstract

The application discloses a training sample generation method and device with labeled random seeds, and belongs to the field of image data processing. The training sample generation method comprises the following steps: selecting and labeling a first preset number of uneven instances from a preset gallery as seeds; randomly sampling the seeds in a uniform distribution manner to generate a second preset number of to-be-processed uneven instances; performing a pretreatment operation on the to-be-processed uneven instances to expand the data quantity; using a Poisson disc region sampling algorithm to generate position coordinates of the to-be-processed uneven instances in a background image, and scattering the position coordinates in the background image to generate label information of a new picture and a corresponding binary mask image; generating a preset number of picture data according to the label information and the binary mask image, and automatically labeling the picture data. The application can help improve the recognition and segmentation capability of an instance segmentation model for uneven small targets, and solve the problems in the prior art.

Description

Technical Field

[0001] This invention relates to the field of image data processing, and in particular to a method, detection method and apparatus for generating training samples with labeled random seeds. Background Technology

[0002] Starch undergoes a transformation process called gelatinization when heated in an aqueous solution. Gelatinization is an important part of the food industry, and various characterization methods are used to study its properties and reveal its underlying mechanisms. During gelatinization, starch gradually swells and decomposes into granular forms of a specific shape. To clearly observe the entire morphological evolution of starch granules, a thermal microscope with precise temperature control is needed to simultaneously monitor the instantaneous appearance and birefringence of the granules within a certain temperature range. The image characteristics of starch granules, such as granule edges, color, and size, change over time. In fact, each stage of starch gelatinization has corresponding characteristics in terms of particle size, color, and edges, which can be used to quickly determine the degree of starch gelatinization. During starch gelatinization, granule edges gradually blur, volume gradually increases, and color gradually becomes more transparent. These three changes occur simultaneously. Therefore, it is necessary to combine these three characteristics to comprehensively evaluate the degree of starch gelatinization. Starch gelatinization is generally divided into three stages: reversible water absorption, irreversible swelling, and complete granule disintegration. Based on the four-dimensional morphological characteristics of particle edge, color, and volume, it can be divided into four stages: initial stage, water absorption and swelling stage, irreversible swelling stage, and disintegration stage.

[0003] Images of starch granules captured under an optical microscope are typically analyzed only through direct visual inspection, or at best, relying on rudimentary image analysis software. Image software analysis depends on manual operation and inevitably introduces subjective errors. Microscopes equipped with heated sections are among the most popular and simplest tools for studying this process in-situ and in real-time under controlled heating conditions (including heating rate and temperature range). For decades, images obtained along the thermal gelatinization process have traditionally been analyzed through direct visual inspection. Because the analysis of gelatinization processes relies heavily on individual subjective skills in various studies, this procedure is difficult to standardize and cannot meet the need for rapid screening of large numbers of samples in a short time, often leading to inconsistent and confusing conclusions. Furthermore, many particles with incomplete or blurred edges exist in the images, making them difficult to detect with the naked eye; therefore, artificial neural networks are needed to identify and capture subtle differences between particles. However, the datasets for starch gelatinization detection experiments are collected manually, incurring huge collection costs, and the dataset size is far from sufficient. To complete this research, a useful and efficient data augmentation method is urgently needed.

[0004] Furthermore, although data augmentation has been widely applied in image classification, object detection, instance segmentation, and speech recognition, traditional data augmentation methods still fail to fully address the challenges in dense, non-uniform instance segmentation tasks. Dense, non-uniform instance segmentation involves numerous small targets with varying shapes. Traditional data augmentation methods primarily provide the entire augmented image to the model for training, rather than the target region. Therefore, small targets are difficult to annotate accurately, ultimately impacting the performance of small target instance segmentation models. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a method, detection method and apparatus for generating training samples with labeled random seeds for dense non-uniform instance segmentation.

[0006] The technical solution adopted in this invention is:

[0007] A method for generating training samples with labeled random seeds includes the following steps:

[0008] Select and label a first preset number of uneven instances from the preset image library as seeds;

[0009] The seeds are randomly sampled in a uniformly distributed manner to generate a second preset number of non-uniform instances to be processed.

[0010] The non-uniform instances to be processed are preprocessed to increase the amount of data; the preprocessing operations include random aspect ratio scaling, mirroring, and rotation.

[0011] Using the Poisson disk region sampling algorithm, the position coordinates of the non-uniform instance to be processed in the background image are generated, and the position coordinates are spread in the background image to generate the label information of the new image and the corresponding binary mask image.

[0012] Generate a preset number of image data based on the label information and binary mask image, and automatically label the image data.

[0013] Furthermore, the preset image library is a database containing multiple images of starch granules;

[0014] The non-uniform instance is any starch granule that needs to be segmented.

[0015] Furthermore, the seeds are highly representative and distinctive starch granules to distinguish different instances and cover a broad range of target classifications and scales to improve the system's generalization ability.

[0016] Furthermore, the uniform distribution refers to random sampling from the seeds, with each seed having an equal probability of being sampled.

[0017] Furthermore, the step of using the Poisson disk region sampling algorithm to generate the position coordinates of the non-uniformity instance to be processed in the background image includes:

[0018] A1. Create a ring-shaped region near the reference particle;

[0019] A2. Randomly generate multiple sampling points within the annular region and determine whether the grid containing each sampling point is occupied by other particles;

[0020] A3. Determine the state of the grid where each sampling point is located. If the state is -1, the particle to be sampled can be directly sampled to that position. If there is a grid with a state other than -1, it is necessary to calculate the DIoU value to determine whether it is possible to sample at that position.

[0021] A4. Traverse all grids whose state is not -1, extract and integrate the particle numbers of these grids, calculate the DIoU value between the particle to be sampled and the corresponding particles of these numbers, and determine whether sampling can be performed at this position.

[0022] A5. Perform steps A3 and A4 on all sampling points respectively until all sampling points have been processed;

[0023] A6. The algorithm ends when the surrounding area is filled with particles or all particles have been sampled.

[0024] Furthermore, the DIoU value refers to the overlap metric of two minimum bounding boxes, which is calculated by comparing the sum of the centroid distance and the diagonal length of the two minimum bounding boxes.

[0025] Furthermore, the size of the annular region is determined by the length and width of the minimum bounding box of the reference particle and the sampled particle.

[0026] Furthermore, the label information of the generated new images is used as training data for instance segmentation, and the corresponding binary mask image is used to evaluate the accuracy of the model.

[0027] Another technical solution adopted in this invention is:

[0028] A method for detecting starch gelatinization includes the following steps:

[0029] Training samples are obtained using the training sample generation method described above, and a training set is constructed.

[0030] The starch gelatinization detection model was trained using a training set;

[0031] The image to be detected is acquired, input into the trained starch gelatinization detection model, and the detection result is output.

[0032] Another technical solution adopted in this invention is:

[0033] An apparatus comprising:

[0034] At least one processor;

[0035] At least one memory for storing at least one program;

[0036] When the at least one program is executed by the at least one processor, the at least one processor performs the method as described above.

[0037] The beneficial effects of this invention are as follows: By selecting representative non-uniform instances as seeds, randomly sampling to generate non-uniform instances to be processed, preprocessing to expand the data volume, using the Poisson disk region sampling algorithm to generate position coordinates, and automatically labeling to generate the dataset, this invention provides a new method and approach for research in the field of non-uniform instance segmentation, and provides an effective means for optimizing instance segmentation models. It can help improve the ability of instance segmentation models to identify and segment non-uniform small targets, solving problems in existing technologies. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the steps in a method for generating training samples with labeled random seeds according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of three preprocessing methods for random shapes in embodiments of the present invention;

[0041] Figure 3 This is a schematic diagram of the random seed position sampling algorithm in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of DIoU in an embodiment of the present invention;

[0043] Figure 5 This is a flowchart of the steps of a starch gelatinization detection method in an embodiment of the present invention. Detailed Implementation

[0044] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0045] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0046] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0047] Furthermore, in the description of this invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0048] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0049] To address the existing technical problems, this application proposes a training sample generation method based on labeled random seeds for dense, non-uniform instance segmentation. This method expands the dataset size and effectively solves the problem of missing target datasets with relatively simple backgrounds and dense, non-uniform shapes. It can generate a large amount of highly realistic new starch particle data and automatically generate annotation files for the new data, effectively improving the scale and quality of the dataset and promoting the development of dense starch particle gelatinization detection.

[0050] like Figure 1As shown, this embodiment provides a method for generating training samples with labeled random seeds, specifically for the detection of dense starch particle gelatinization. This method can generate labeled data samples that can be applied to most existing CNN models. This method can help improve the model's ability to recognize and segment small objects, solving problems in existing technologies. The method includes the following steps:

[0051] S1. Select and label a first preset number of uneven instances from the preset image library as seeds.

[0052] First, users can select and label a small number of representative starch granules from the image library as seeds, or they can use the system's built-in starch granule library as a seed dataset.

[0053] S2. Randomly sample the seeds in a uniform distribution to generate a second preset number of non-uniform instances to be processed.

[0054] Then, the system will randomly sample these seeds in a uniform distribution to generate a certain number of non-uniform instances to be processed.

[0055] S3. Perform preprocessing operations on the uneven instances to be processed to increase the amount of data; wherein the preprocessing operations include random aspect ratio scaling, mirroring, and rotation.

[0056] Next, the system will perform preprocessing operations such as random aspect ratio scaling, mirroring, and rotation on these instances of uneven data to increase the amount of data.

[0057] S4. Using the Poisson disk region sampling algorithm, generate the position coordinates of the non-uniform instance to be processed in the background image, and spread the position coordinates in the background image to generate the label information of the new image and the corresponding binary mask image.

[0058] Subsequently, the system will use the Poisson disk region sampling algorithm to generate the position coordinates of these dense and non-uniform instances to be processed in the background image, and then spread them in the background image. At the same time, it will generate the label information of the new image and the corresponding binary mask image.

[0059] S5. Generate a preset number of image data based on the label information and binary mask image, and automatically label them. Here, "automatically label" refers to automatically generating new labeled image data.

[0060] Finally, the system will generate the required amount of image data as needed and automatically label it.

[0061] The above method will be explained in detail below with reference to the accompanying drawings and specific embodiments.

[0062] This embodiment provides a random seed-based data augmentation algorithm for augmenting dense, non-uniform instance data, as follows:

[0063] S101. Select several starch granules as seeds, including 16 primary granules, 23 secondary granules, 19 tertiary granules and 24 quaternary granules. These seeds need to cover various features of target granules in the four stages. Then use labelme to perform pixel-level edge annotation and classification annotation on these seeds.

[0064] S102. These seeds are randomly sampled in a uniform distribution, and 50-100 particles are randomly generated by setting a random number n.

[0065] S103. Perform preprocessing operations such as random aspect ratio scaling, random mirroring, and random rotation on the selected seed with a certain probability.

[0066] S104. After processing n particles, use our Poisson disk region sampling algorithm to randomly generate the position coordinates of n particles in the background image, representing the position of the seed center in the background image. Spread the n particles in the background image and generate the label information of the new image and the corresponding binary mask image.

[0067] After generating a new image, label data with pixel-level edge annotations and binary mask images is automatically generated for these new images. The randomness of the random seed data augmentation is reflected in the random targets, random shapes, and random locations.

[0068] The first part consists of random targets. The algorithm pre-selects particles that can cover all features as seeds, and then randomly selects them using uniform sampling. The number of targets in each image is also random.

[0069] The second part is random shape, which uses random aspect ratio scaling, random flipping, and random rotation to perform random shape transformation on the selected seed. In the third step of seed preprocessing, we use matrix affine transformation to process the seed edge position transformation. Formulas (1), (2), and (3) represent the affine transformation algorithms for random aspect ratio scaling, random mirroring, and random rotation, respectively.

[0070] (1) Random aspect ratio scaling

[0071]

[0072] In formula (1), x,y and x1,y1 represent the coordinates of each point on the seed edge before and after aspect ratio scaling transformation, respectively, and x_scale and y_scale represent aspect ratio scaling ratio.

[0073] (2) Random mirroring

[0074] In formula (2), x1,y1 and x2,y2 represent the coordinates of a point on the edge of the seed after aspect ratio scaling transformation and before and after horizontal mirror transformation, respectively, and w_seed represents the width of the seed.

[0075]

[0076] (3) Random rotation

[0077] The seed is randomly rotated at an angle θ in the range of [-180°, 180°], and then scaled proportionally with a scaling factor of scale. First, the center of the seed is moved to the origin of the coordinate axis; then the seed is rotated counterclockwise around the origin by an angle θ. After the rotation, the new width (w) and height (h) of the seed are calculated, as shown in formula (3), where w_seed and h_seed represent the width and height of the seed before the rotation, respectively; finally, the seed is moved by a length of w / 2 along the positive direction of the horizontal axis and by a length of h / 2 along the positive direction of the vertical axis, respectively. In formula (3), x2,y2 and x3,y3 represent the coordinates of a point on the edge of the seed after the random mirror transformation before and after the counterclockwise rotation θ, respectively, and the round() function represents the rounding operation.

[0078]

[0079] in,

[0080] The third part involves random positioning. A Poisson disk region sampling algorithm is designed, starting with the randomly obtained seed position in the background image, to achieve a uniform and non-overlapping distribution of particles of varying sizes and shapes. The algorithm is characterized by: a) the sampling point determines the center position of the particle, thus determining the particle's location; b) requiring particles to not overlap. During each sampling, the distance standard between the particle's center point and existing points is dynamically changing; therefore, the system designs an algorithm that can quickly generate non-overlapping particle positions.

[0081] To generate n particles from the background image, first, use the various preprocessing methods described earlier to randomly generate n particles. Store the coordinates of their center points and the length and width of their minimum bounding boxes in a one-dimensional array `particle`, where the array index represents the particle number. Design a pointer `index` to point to the particle to be sampled in the `particle` array. When the particle number is less than `index`, it means that the particle's position on the background image has been determined, and the particle sampling is complete; if the particle number is greater than `index`, it means that the particle has not yet been sampled. Design a pointer `head` to point to the reference particle in the `particle` array. Each time, only search for a particle position that meets the requirements near the reference particle; the reference particle has definitely been sampled. Divide the w_bg×h_bg background image into 50×50 equal parts using grid lines. The size of each grid is (w_bg / 50×h_bg / 50). Design a 50×50 two-dimensional array grid to represent the grid state. The following points should be noted about grid: 1) If the value of grid[i][j] is -1, it means that the grid has not been occupied by any particles. The initial value of grid is -1. 2) A particle may occupy multiple grids at the same time. Each time a particle position is sampled, the particle number is added to grid[i][j] and -1 is deleted (if -1 exists). 3) A grid may be occupied by multiple particles at the same time. If there are multiple integer numbers in grid[i][j], it means that the grid is occupied by particles with these numbers at the same time. Set a DIoU threshold α. If the DIoU value between the minimum bounding boxes of two particles is greater than this threshold α, it is considered that the two particles are too close or overlap. If the DIoU value between the two minimum bounding boxes is less than this threshold α, it means that the positional relationship between the two particles meets the requirements.

[0082] During the first sampling, the coordinate position in the background image is randomly generated, and particle number 0 is scattered at that position, thus completing the sampling of particle number 0. The change of target edge coordinate information during the sowing process is shown in formula (4), where w and h are the width and height of the seed in formula (3), (x0, y0) is the position coordinate of the seed in the background image, (x3, y3) is obtained from formula (3), and (x, y) is the coordinate of any point on the seed edge in the background image.

[0083]

[0084] Simultaneously, the states of all grid cells occupied by particle 0 are set to 0, particle 0 is used as the reference point, and head points to particle 0. Each sampling follows these steps:

[0085] 1) Continue to search for points that meet the requirements near the head of the reference particle. w1 and h1 represent the length and width of the minimum bounding box of the reference particle, and w2 and h2 represent the length and width of the minimum bounding box of the index of the particle to be sampled. Design an annulus with r1 as the inner radius, r2 as the outer radius, and the center point of the reference particle as the center. Randomly sample 20 points within this annulus. The definitions of r1 and r2 are shown in formula (5).

[0086]

[0087] 2) Check if the grid state of each of the 20 points is -1. If it is not -1, it means that the grid has been occupied by other particles, and the sampling point is discarded. Note that if all the grids containing the sampling points are occupied, only the last sampling point is kept, and all others are discarded. Finally, one or more sampling points will be obtained and stored in a one-dimensional array `dot`.

[0088] The calculation of any point (x,y) on the background image falling into the cell of the p+1th row and q+1th column is shown in formula (6). w_bg and h_bg are the width and height of the background image, respectively. N is equal to 50 here, which means that the background image is divided into N×N cells. w_s and h_s represent the width and height of each small cell.

[0089]

[0090] Among them, w_s=w_bg / N, h_s=h_bg / N

[0091] 3) Spread the particle index to be sampled to the first sampling point and extract the state of all grids occupied by the particle index. If the state of all grids is -1, it means that no particles currently occupy these grids. These grid areas are "safe" areas, and the particle index to be sampled can be directly sampled to this position, with the index pointing to the next particle.

[0092] 4) If a grid cell is occupied by other particles, search through these grid cells whose states are not -1, extract and integrate the particle numbers of these grid cells, and store them in a one-dimensional array `compare`. Calculate the DIoU value between the sampled particle `index` and the numbered particles in the `compare` array. If all DIoU values ​​are less than or equal to the threshold α, place the sampled particle `index` at that position and add the number of the sampled particle `index` to all states of the occupied grid cell, with `index` pointing to the next particle. If there is a DIoU value greater than the threshold α, consider this position unqualified and discard this sampling point.

[0093] 5) Process all sampling points in the dot sequentially using steps 3) and 4).

[0094] 6) After processing all the sampling points in the dot, we assume that the area around the reference point head is full of particles. We then replace the next particle with the reference particle, and the head points to the next particle. We then repeat steps 1), 2), 3), 4), and 5).

[0095] 7) When all particles in the particle have completed sampling, i.e., when the index points to the empty particle next to the last particle, or when all currently sampled particles have been used as reference particles and there are no more sampled particles that can be used as reference particles, i.e., when the head is greater than or equal to the index, the loop terminates, indicating that all sampling has been completed.

[0096] DIoU (Distance Intersection over Union) is shown in formula (7), where d represents the Euclidean distance between the center points of the two boxes, and c represents the diagonal distance of the smallest enclosing region that can simultaneously contain both boxes. DIoU integrates the distance information between the two boxes, using the inter-box distance as a penalty term to reduce the occurrence of False Positives or False Negatives.

[0097]

[0098] Accordingly, an automatic annotation function is provided to automatically generate a dense, non-uniform instance dataset with labels according to user needs for training of instance segmentation tasks.

[0099] Example 2

[0100] This embodiment provides a method for generating labeled random seeds based on dense non-uniform instance segmentation, including the following steps:

[0101] Step 1: Seed selection.

[0102] Users select and label a small number of representative starch granules from their personal datasets or image libraries as seeds, and need to ensure the representativeness and integrity of the seeds as much as possible.

[0103] Step 2: Non-uniform instance generation.

[0104] After selecting and labeling the seeds, non-uniform instances are automatically generated by performing preprocessing operations such as random aspect ratio scaling, mirroring, and rotation on the seeds to expand the data volume. The generation of non-uniform instances is one of the core parts of this invention, aiming to expand the seeds into more instances and broaden the instance distribution to handle unevenly distributed instances. The specific operations are as follows:

[0105] The algorithm selects random targets by first choosing particles that can cover all features as seeds, and then randomly selects them using uniform sampling. The number of targets in each image is also random.

[0106] Select a random shape and perform random shape transformation on the selected seed using random aspect ratio scaling, random flipping, and random rotation. In the third step of seed preprocessing, we use matrix affine transformation to process the seed edge position transformation. Formulas (8), (9), and (10) represent the affine transformation algorithms for random aspect ratio scaling, random mirroring, and random rotation, respectively.

[0107] (1) Random aspect ratio scaling

[0108]

[0109] like Figure 2 As shown in the first row of the diagram, the seed is scaled up by a certain ratio in both the horizontal and vertical directions. In formula (8), x,y and x1,y1 represent the coordinates of each point on the seed edge of the seed image before and after the aspect ratio scaling transformation, respectively, and x_scale and y_scale represent the aspect ratio scaling ratio.

[0110] (2) Random mirroring

[0111] Taking horizontal mirroring as an example, such as Figure 2 As shown in the second row of the diagram, the seed is mirrored on the horizontal axis and then horizontally shifted to the right by the width of the seed. In formula (9), x1,y1 and x2,y2 represent the coordinates of a point on the edge of the seed before and after the horizontal mirror transformation, respectively, and w_seed represents the width of the seed.

[0112]

[0113] (3) Random rotation

[0114] Rotate the seed randomly by an angle θ within the range of [-180°, 180°], then scale the seed proportionally with a scaling factor of scale. For example... Figure 2 As shown in the diagram and formula (10) in the third row, the seed center is first moved to the origin of the coordinate axis; then the seed is rotated counterclockwise around the origin by an angle θ. After the rotation, the new width (w) and height (h) of the seed are calculated, as shown in formula (10). w_seed and h_seed represent the width and height of the seed before the rotation, respectively. Finally, the seed is moved by a length of w / 2 along the positive direction of the horizontal axis and by a length of h / 2 along the positive direction of the vertical axis. In formula (10), x2,y2 and x3,y3 represent the coordinates of a point on the edge of the seed after the random mirror transformation before and after the counterclockwise rotation θ, respectively. The round() function represents the rounding operation.

[0115]

[0116] in,

[0117] Step 3: Use a random seed location sampling algorithm, specifically the Poisson disk region sampling algorithm, to generate the position coordinates of these dense, non-uniform instances to be processed in the background image, and then spread them across the background image. Simultaneously, generate label information for the new image and the corresponding binary mask. The random seed location sampling algorithm starts from a single point and gradually expands outwards, ensuring that particles of different sizes and shapes are evenly and non-overlappingly distributed on the background image. Its characteristics are: a) The sampled point represents the center position of the particle; once the point's position is determined, the particle's position is also determined. b) The particles vary in size and shape, requiring them to not overlap. Since the distance standard between the particle's center point and existing points changes dynamically with each sampling, an algorithm for quickly generating non-overlapping particle positions is designed in the system.

[0118] To generate n particles from the background image, first, use the various preprocessing methods described earlier to randomly generate n particles. Store the coordinates of their center points and the length and width of their minimum bounding boxes in a one-dimensional array `particle`, where the array index represents the particle number. Design a pointer `index` to point to the particle to be sampled in the `particle` array. When the particle number is less than `index`, it means that the particle's position on the background image has been determined, and the particle sampling is complete; if the particle number is greater than `index`, it means that the particle has not yet been sampled. Design a pointer `head` to point to the reference particle in the `particle` array. Each time, only search for a particle position that meets the requirements near the reference particle; the reference particle has definitely been sampled. Divide the w_bg×h_bg background image into 50×50 equal parts using grid lines. The size of each grid is (w_bg / 50×h_bg / 50). Design a 50×50 two-dimensional array grid to represent the grid state. The following points should be noted about grid: 1) If the value of grid[i][j] is -1, it means that the grid has not been occupied by any particles. The initial value of grid is -1. 2) A particle may occupy multiple grids at the same time. Each time a particle position is sampled, the particle number is added to grid[i][j] and -1 is deleted (if -1 exists). 3) A grid may be occupied by multiple particles at the same time. If there are multiple integer numbers in grid[i][j], it means that the grid is occupied by particles with these numbers at the same time. Set a DIoU threshold α. If the DIoU value between the minimum bounding boxes of two particles is greater than this threshold α, it is considered that the two particles are too close or overlap. If the DIoU value between the two minimum bounding boxes is less than this threshold α, it means that the positional relationship between the two particles meets the requirements.

[0119] During the first sampling, the coordinate position in the background image is randomly generated, and particle number 0 is scattered at that position, thus completing the sampling of particle number 0. The change of target edge coordinate information during the sowing process is shown in formula (11), where w and h are the width and height of the seed image in formula (10), (x0, y0) are the position coordinates of the seed in the background image, (x3, y3) are obtained from formula (10), and (x, y) are the coordinates of any point on the seed edge in the background image.

[0120]

[0121] Simultaneously, set the state of all grid cells occupied by particle 0 to 0, use particle 0 as the reference point, and point the head towards particle 0. For example... Figure 3 As shown in the first row, the green box represents the smallest bounding box of particle 0. The state of the grid region containing it is all set to 0, indicating that this cell is occupied by particle 0. Subsequent sampling follows these steps:

[0122] 1) Continue searching for points that meet the requirements near the reference particle head. w1 and h1 represent the length and width of the minimum bounding box of the reference particle, and w2 and h2 represent the length and width of the minimum bounding box of the sampled particle index. Design an annulus with r1 as the inner radius, r2 as the outer radius, and the center point of the reference particle as the center. Randomly sample 20 points within this annulus. The definitions of r1 and r2 are shown in formula (12). Figure 3 As shown in Figures (a), (b), and (c), points are randomly selected within the blue dashed circle.

[0123]

[0124] 2) Sequentially check if the grid state of each of these 20 points is -1. If it is not -1, it means that the grid has been occupied by other existing particles, so discard the sampling point. For example... Figure 3 As shown in Figure (a), the yellow box represents the minimum bounding box of the particle to be sampled. If the center point of the represented position falls on a cell whose state is not -1, the position is discarded; otherwise, it is discarded. Figure 3 As shown in figures (b) and (c), this sampling location proceeds to the next step. It's important to note that if all grid points are occupied, only the last sampling point is retained, and the others are discarded. Ultimately, one or more sampling points will be obtained and stored in a one-dimensional array `dot`.

[0125] The calculation of any point (x, y) on the background image falling into the cell of row p+1 and column q+1 is shown in formula (13). w_bg and h_bg are the width and height of the background image, respectively. N is equal to 50 here, which means that the background image is divided into N×N cells. w_s and h_s represent the width and height of each small cell.

[0126]

[0127] Among them, w_s=w_bg / N, h_s=h_bg / N

[0128] 3) Spread the particle index to be sampled to the first sampling point, and extract the state of all grids occupied by the particle index. If the state of all grids is -1, it means that no particles currently occupy these grids, and these grid areas are "safe" areas. The particle index can be directly sampled to this position, and the index points to the next particle. Figure 3 As shown in Figure (c), the generated sampled particles do not overlap with any of the existing particles.

[0129] 4) If a grid cell is occupied by other particles, search through these grid cells whose states are not -1, extract and integrate the particle numbers of these grid cells, and store them in a one-dimensional array `compare`. Calculate the DIoU value between the sampled particle `index` and the numbered particles in the `compare` array. If all DIoU values ​​are less than or equal to the threshold α, place the sampled particle `index` at that position and add its number to all states of the occupied cell, with `index` pointing to the next particle. If there is a DIoU value greater than the threshold α, consider that position unqualified and discard the sampling point. Figure 2 If the DIoU value between the yellow minimum bounding box of the sampled particle shown in Figure (b) and all minimum bounding boxes in the array compare is less than the threshold α, then the sampled particle will be seeded at that location.

[0130] 5) Process all sampling points in the dot sequentially using steps 3) and 4).

[0131] 6) After processing all the sampling points in the dot, we assume that the area around the reference point head is full of particles. We then replace the next particle with the reference particle, and the head points to the next particle. We then repeat steps 1), 2), 3), 4), and 5).

[0132] 7) When all particles in the particle have completed sampling, i.e., when the index points to the empty particle next to the last particle, or when all currently sampled particles have been used as reference particles and there are no more sampled particles that can be used as reference particles, i.e., when the head is greater than or equal to the index, the loop terminates, indicating that all sampling has been completed.

[0133] Where DIOU (Distance Intersection over Union) is as shown in formula (14) and Figure 4 As shown, d represents the Euclidean distance between the center points of the two bounding boxes, and c represents the diagonal distance of the smallest closure region that can simultaneously contain both boxes. DIoU integrates the distance information between the two boxes, using the inter-box distance as a penalty term to reduce the occurrence of FalsePositive or FalseNegative errors. DIoU does not require calculating the area of ​​the smallest convex polygon, making it computationally simpler than GIoU. Furthermore, DIoU can effectively handle situations where the actual distance between the two boxes is relatively large, improving the accuracy and robustness of the detection model.

[0134]

[0135] The Poisson disk region sampling algorithm can complete the sampling of a region block in O(n) time and O(n) memory. The pseudocode for the entire random seed data augmentation algorithm process is shown in Table 1.

[0136] Table 1 Random Seed Data Augmentation Algorithm

[0137]

[0138]

[0139] In summary, this embodiment proposes a labeled random seed generation method based on dense non-uniform instance segmentation, which can help improve the ability of instance segmentation models to identify and segment non-uniform small targets, solving the problems in existing technologies. This system selects representative non-uniform instances as seeds, randomly samples to generate non-uniform instances to be processed, preprocesses to expand the data volume, uses the Poisson disk region sampling algorithm to generate position coordinates, and automatically labels to generate the dataset. It provides new methods and ideas for research in the field of non-uniform instance segmentation and offers an effective means for optimizing instance segmentation models.

[0140] like Figure 5 As shown, based on the above-described method for generating training samples with labeled random seeds, this embodiment also provides a method for detecting starch gelatinization, including the following steps:

[0141] S501, via such Figure 1 The training sample generation method shown obtains training samples and constructs a training set;

[0142] S502. The starch gelatinization detection model is trained using a training set;

[0143] S503. Obtain the image to be detected, input the image into the trained starch gelatinization detection model, and output the detection result.

[0144] This embodiment also provides an apparatus, including:

[0145] At least one processor;

[0146] At least one memory for storing at least one program;

[0147] When the at least one program is executed by the at least one processor, the at least one processor performs the following: Figure 1 and Figure 5 The method shown.

[0148] The apparatus of this embodiment can execute the training sample generation method with labeled random seeds and the starch gelatinization detection method provided in the method embodiment of the present invention. It can execute any combination of the implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0149] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 1 and Figure 5 The method shown.

[0150] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0151] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0152] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0154] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0155] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0156] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0157] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0158] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for generating a training sample with a labeled random seed, characterized in that, The method comprises the following steps: selecting and labeling a first preset number of uneven instances from a preset gallery as seeds; randomly sampling the seeds in a uniform distribution manner to generate a second preset number of to-be-processed uneven instances; performing a preprocessing operation on the to-be-processed uneven instances to expand the data volume; wherein the preprocessing operation comprises random aspect ratio scaling, mirroring and rotation; using a Poisson disc area sampling algorithm to generate position coordinates of the to-be-processed uneven instances in a background image, and to scatter the position coordinates in the background image to generate label information of a new picture and a corresponding binary mask image; generating a preset number of picture data according to the label information and the binary mask image, and automatically labeling the picture data; the step of using the Poisson disc area sampling algorithm to generate the position coordinates of the to-be-processed uneven instances in the background image comprises: A1, creating a ring region around a reference particle; A2, randomly generating a plurality of sampling points in the ring region, and judging whether the grid where each sampling point is located is occupied by other particles; A3, judging the state of the grid where each sampling point is located, if the state is -1, the sampling particle can be directly sampled to the position, if there is a grid with a state not equal to -1, the DIoU value needs to be calculated to judge whether the particle can be sampled at the position; A4, traversing all grids with a state not equal to -1, extracting and integrating the particle numbers of these grids, calculating the DIoU value between the sampling particle and the particles corresponding to these numbers, and judging whether the particle can be sampled at the position; A5, performing steps A3 and A4 on all sampling points until all sampling points are processed; A6, when the surrounding particles are full or all particles are sampled, the algorithm ends.

2. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The preset gallery is a database containing a plurality of starch granule pictures; The uneven instance is any starch granule that needs instance segmentation.

3. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The seeds are representative and high-differentiation starch granules, which can distinguish different instances and cover a wide range of target categories and scales to improve the generalization ability of the system.

4. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The uniform distribution manner means that the seeds are randomly sampled with the same probability.

5. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The DIoU value refers to the overlap measure of two minimum bounding boxes, which is calculated by comparing the sum of the center distance and the diagonal length of the two minimum bounding boxes.

6. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The size of the ring region is determined by the length and width of the minimum bounding box of the reference particle and the sampling particle.

7. The training sample generation method with labeled random seeds according to claim 1, characterized in that, The label information of the generated new picture is used as the training data for instance segmentation, and the corresponding binary mask image is used to evaluate the accuracy of the model.

8. A method of detecting starch gelatinization, characterized by, The method comprises the following steps: obtaining training samples by the training sample generation method according to any one of claims 1-7, and constructing a training set; training a starch gelatinization detection model using the training set; obtaining an image to be detected, inputting the image into the trained starch gelatinization detection model, and outputting a detection result.

9. A labeled random seed training sample generation apparatus, characterized by, It comprises: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1-7.

Images