A sample image processing method and related device

Abstract

The present application discloses a method for processing sample images and a related device, the method comprising: determining image size reference information based on size information of each sample image in a sample image group; determining the number of targets corresponding to each sample image respectively, the number of targets being the number of targets contained in the corresponding sample image; using the determined number of targets and image size reference information, cropping each sample image respectively to obtain a local image corresponding to each sample image; splicing each local image obtained by cropping to obtain an extended image and output it. The method provided by the present application can reduce the amount of useless information generated in the sample image and increase the frequency of occurrence of the target frame.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a sample image processing method and related devices. Background Art

[0002] Object detection is one of the most widely used tasks in computer vision, playing an extremely important role in fields such as traffic control, video surveillance, and autonomous driving. Currently, mosaic data augmentation is widely used to obtain training samples for object detection tasks, which is used to improve the detection performance of object detection models. However, the images generated by mosaic data augmentation have the same width and height values. Therefore, in order to make the generated images consistent with the size of the video images, padding operations are often required before they can be output as training samples. However, excessive padding operations will generate a large amount of useless information, reduce the frequency of target boxes, increase the training cost of the model, and reduce the speed of model training. Therefore, a technical solution is urgently needed to solve these technical problems. Summary of the Invention

[0003] The main technical problem solved by this application is to provide a sample image processing method and related devices, which can reduce the amount of useless information generated in the sample image and increase the frequency of target frame appearance.

[0004] To solve the above technical problems, a technical solution adopted in this application is to provide a method for processing a sample image, the method comprising:

[0005] Determining image size reference information based on size information of each sample image in the sample image group;

[0006] Determining the number of targets corresponding to each sample image respectively, where the number of targets is the number of targets contained in the corresponding sample image;

[0007] Using the determined number of targets and the image size reference information, cropping each sample image to obtain a local image corresponding to each sample image;

[0008] The partial images obtained by stitching and cropping are combined to obtain an extended image and output it.

[0009] To solve the above technical problems, another technical solution adopted by the present application is to provide a method for processing a sample image, the method comprising:

[0010] For each target frame included in each extended image, selecting a first-category anchor frame corresponding to the target frame based on a size ratio between the target frame and each anchor frame, wherein the extended image is obtained based on any of the above methods;

[0011] For each of the target boxes, a similarity ratio between each of the first-category anchor boxes and the target box is calculated, and the first-category anchor boxes whose similarity ratio is greater than or equal to a first threshold are output as positive samples of the target box.

[0012] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide an electronic device, the electronic device comprising a processor and a memory coupled to the processor; wherein,

[0013] The memory is used to store computer programs;

[0014] The processor is configured to run the computer program to perform any one of the methods described above.

[0015] To solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that can be executed by a processor, and the computer program is used to implement any of the methods described above.

[0016] The beneficial effects of the present application are as follows: Different from the existing technology, the technical solution provided by the present application determines the image size reference information based on the size information of each sample image in the sample image group, and then determines the number of targets corresponding to each sample image. Then, using the determined number of targets and the image size reference information, each sample image is cropped to obtain a local image corresponding to each sample image; finally, the cropped local images are spliced ​​together to obtain an extended image and output. In the technical solution provided by the present application, by combining the number of targets included in the sample image to obtain a local image from each sample image, and then correspondingly splicing the resulting local images, the amount of useless information generated in the sample image is reduced, while the frequency of target frame appearance is increased, achieving a good technical effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of an embodiment of a method for processing a sample image of the present application

[0018] Figure 2 This is a flow chart of another embodiment of a method for processing a sample image of the present application;

[0019] Figure 3 This is a flow chart of another embodiment of a method for processing a sample image of the present application;

[0020] Figure 4 This is a schematic diagram of an application scenario in an embodiment of a method for processing a sample image of the present application;

[0021] Figure 5 This is a flow chart of another embodiment of a method for processing a sample image of the present application;

[0022] Figure 6 This is a schematic diagram of an application scenario in an embodiment of a method for processing a sample image of the present application;

[0023] Figure 7 This is a flow chart of another embodiment of a method for processing a sample image of the present application;

[0024] Figure 8 This is a flow chart of an embodiment of a method for processing a sample image according to the present application;

[0025] Figure 9 This is a flow chart of another embodiment of a method for processing a sample image of the present application;

[0026] Figure 10 This is a structural diagram of an electronic device in an embodiment of the present application;

[0027] Figure 11 This is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically limited. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to the process, method, product, or device.

[0030] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] See Figure 1 , Figure 1 FIG. 1 is a flow chart of an embodiment of a method for processing a sample image of the present application. In the current embodiment, the method provided by the present application includes steps S110 to S140.

[0032] S110: Determine image size reference information based on size information of each sample image in the sample image group.

[0033] In the current embodiment, when processing sample images to obtain expanded images for model training, image size reference information is first determined based on the size information of each sample image included in the sample image group. The sample image group includes a first preset number of sample images, and the image size reference information is the size information used as a reference when processing the sample images included in the current sample image group.

[0034] Furthermore, in one embodiment, the size information of the sample image includes the width and height of the sample image. Correspondingly, the image size reference information includes the average width and average height of the sample images included in the sample image group.

[0035] S120: Determine the number of targets corresponding to each sample image.

[0036] When determining the image size reference information, the number of objects included in each sample image included in the sample image group is also determined, wherein the number of objects is the number of objects included in the corresponding sample image.

[0037] S130: Using the determined number of objects and image size reference information, crop each sample image to obtain a local image corresponding to each sample image.

[0038] After determining the image size reference information and the number of targets corresponding to each sample image, each sample image is further cropped using the determined number of targets and image size reference information to obtain a partial image corresponding to each sample image. A partial image refers to a piece of image cropped from a sample image and used to stitch together the expanded image.

[0039] Furthermore, step S130 includes: determining the interception point coordinates and insertion point coordinates of each sample image based on the image size reference information and the number of objects corresponding to each sample image; and then cropping the corresponding sample image to obtain a partial image based on the interception point coordinates of each sample image. For detailed description, please refer to the following Figure 2 The corresponding embodiment.

[0040] S140: stitching the cropped partial images to obtain an extended image and output it.

[0041] After the partial images are obtained by cropping, the cropped partial images are further spliced ​​to obtain an extended image, and the spliced ​​extended image is output.

[0042] This application Figure 1 In the corresponding embodiment, image size reference information is determined based on the size information of each sample image in the sample image group, and then the number of targets corresponding to each sample image is determined. The determined number of targets and the image size reference information are then used to crop each sample image to obtain a local image corresponding to each sample image. Finally, the cropped local images are spliced ​​together to obtain an expanded image and output. In the technical solution provided in this application, by cropping local images from each sample image based on the number of targets included in the sample image, and then stitching the resulting local images together, the amount of useless information generated in the sample images is reduced, while the frequency of target frame appearance is increased.

[0043] See Figure 2 , Figure 2 FIG2 is a flow chart of another embodiment of a method for processing a sample image of the present application. In the current embodiment, the technical solution provided by the present application includes steps S210 to S260.

[0044] S210: traversing the sample training set to select a first-category image, and randomly selecting a second-category preset number of second-category images from the remaining sample images in the sample training set, and outputting the first-category image and the second-category image as a sample image group.

[0045] In the current embodiment, before determining the image size reference information corresponding to the sample image group, it is first necessary to determine the sample image group from the sample training set. Among them, a number of sample images are pre-stored in the sample training set. There is no limit on the number of sample images stored in the sample training set, and the specific number is based on the actual setting. At the same time, the sample image can also be understood as an image that has not been subjected to sample enhancement processing, and the sample image includes targets. There is no limit on the types and number of targets included in the sample image. Furthermore, the types and number of targets included in the sample image can be determined according to the actual application scenario of the trained model. For example, if the extended image obtained by processing is used for training and the target detection model obtained is used for road traffic, then the targets in the sample image may include various types of targets that may be included in the road traffic scenario.

[0046] The sample image group includes a first preset number of sample images, and the first preset number of sample images includes one image of the first category and a second preset number of images of the second category. The first preset number can be set and adjusted based on actual needs, and the first preset number is an even number. In one embodiment, the first preset number can be set to 2. In another embodiment, the first preset number can be set to 4. In yet another embodiment, the first preset number can be set to 6 or 8.

[0047] After determining a sample image group in the sample training set, image size reference information for the obtained sample image group is further determined. In one embodiment, the average width and average height of the sample images included in the sample image group are obtained. Specifically, the sum of the widths of the sample images in the sample image group is first calculated, and then the calculated sum of the widths is divided by a first preset number to obtain the average width. Similarly, the sum of the heights of the sample images in the sample image group is divided by the first preset number to obtain the average height.

[0048] It should be noted that each sample image in the training sample set is only selected as the first category image once, and the sample images in the training set can be selected as the second category images multiple times. In the process of splicing the sample images in the sample training set, each sample image included in the sample training set will be traversed and used as the first category image respectively. For example, the training sample set includes 100 sample images, and each of these 100 sample images will be selected as the first category image respectively. When selecting the second category image, it is randomly selected from the remaining sample images excluding the current first category image. Taking the training sample set as an example, which includes 100 sample images and the first preset number is 4, the 100 sample images are traversed in turn, one image is selected from the 100 images as the first category image, and 3 images are randomly selected from the 99 images excluding the current first category image as the second category images.

[0049] S220: Determine image size reference information based on size information of each sample image in the sample image group.

[0050] Step S220 is the same as step S110 described above, and for details, please refer to the corresponding part above.

[0051] S230: Determine the number of targets corresponding to each sample image.

[0052] For each sample image included in the sample image group, the number of objects included in each sample image is determined. For example, in one embodiment, if four images are obtained from the sample training set in step S210, the number of objects included in each of the four obtained images is determined in step S230.

[0053] Furthermore, in one embodiment, each sample image stored in the training sample set is pre-marked with information about the targets included therein, wherein the target information includes at least the type of target and the target frame, and step S230 can be to directly count the number of targets included in each sample image based on the target frame. Furthermore, when each target frame and each target are in a one-to-one correspondence, the number of targets included in each sample image can be determined by counting the number of target frames. Still further, in another embodiment, when each target frame and the target can be in a one-to-many relationship, the number of targets included in the target frame and the position information of the target frame's marking points will be pre-marked in the target information corresponding to each target frame, so the number of targets included in each sample image can be determined directly based on the number of targets marked on each target frame included in each sample image.

[0054] Furthermore, it should be noted that it is not limited here to executing step S220 first and then executing step S230. In some embodiments, step S230 may also be executed while executing step S220.

[0055] Furthermore, in another embodiment, before determining the number of targets included in each sample image in the above-mentioned step S230 (in other embodiments, the following steps may also be performed before cropping each sample image separately using the determined number of targets and the image size reference information), the method provided in the present application also includes: for each sample image, determining the scaling ratio of each sample image based on the width and average height of the sample image, or the height and average height of the sample image; and scaling each image to be processed according to the scaling ratio.

[0056] Furthermore, the scaling ratio is determined based on the longer side of the sample image. For example, if a sample image is wider than its length, i.e., the longer side of the sample image is the width side, the scaling ratio is further determined based on the width of the sample image and the average width of the sample image group: scaling ratio = average width / sample image width.

[0057] It should be noted that for each sample image in the sample image group, the scaling ratio is determined based on the longer side of each sample image. For example, if the first preset number is 4, and the longer sides of three sample images (a, b, and c) are in the width direction, and the longer side of one sample image (d) is in the height direction, then the scaling ratios of sample images a, b, and c are calculated based on the widths of sample images a, b, and c and the average width of the first preset number of sample images, respectively. The scaling ratio of sample image d is calculated based on the height of sample image d and the average height of the first preset number of sample images. After the scaling ratios of each sample image are determined, each sample image is scaled based on the determined scaling ratios, i.e., sample images a, b, c, and d are scaled based on the scaling ratios of sample images a, b, c, and d.

[0058] S240: Determine the coordinates of the interception point and the insertion point of each sample image based on the image size reference information and the number of objects corresponding to each sample image.

[0059] The image size reference information includes the average width and average height. Therefore, after determining the average width and average height of the sample image group and the number of objects included in each sample image, the coordinates of the interception point and the insertion point of each sample image are further determined based on the determined average width, average height, and number of objects included in each sample image.

[0060] Among them, the interception point coordinates refer to the coordinates of the intercepted partial image in the sample image when the partial image is intercepted in the sample image; the insertion point coordinates refer to the coordinates of the partial image in the stitching reference frame when the intercepted partial image is inserted into the stitching reference frame.

[0061] Specifically, the insertion point coordinates may refer to the coordinates of vertices on a set diagonal of the partial image when the partial image is inserted into the stitching reference frame. The interception point coordinates may refer to the coordinates of vertices on the set diagonal of the intercepted partial image in the sample image. For the same partial image in the same sample image, the insertion point coordinates and interception point coordinates refer to the same set diagonal.

[0062] It should be noted that the interception point coordinates refer to the coordinates of the vertices on the diagonal line of the partial image in the first coordinate system, and the insertion point coordinates refer to the coordinates of the vertices on the diagonal line when the partial image is inserted into the stitching reference frame in the second coordinate system. The first coordinate system is a coordinate system established based on the sample image, the second coordinate system is a coordinate system established based on the stitching reference frame, and the first coordinate system and the second coordinate system are different coordinate systems. In another embodiment, the first coordinate system and the second coordinate system can also be set as the same coordinate system, that is, the sample image and the stitching reference frame are placed in the same quadrant of the same coordinate system. It should be noted that the zero point of the first coordinate system is set at a vertex of the sample image, and the vertices on the same diagonal line as the zero point are located in the first quadrant of the first coordinate system. The zero point of the second coordinate system is set at a vertex of the stitching reference frame, and the vertices in the stitching reference frame that are on the same diagonal line as the zero point of the second coordinate system are located in the first quadrant of the second coordinate system.

[0063] In one embodiment, the insertion point coordinates may be the coordinates of the upper left vertex and the lower right vertex of the partial image in the stitching reference frame when the partial image is inserted into the stitching reference frame. Correspondingly, the interception point coordinates are the coordinates of the upper left vertex and the lower right vertex of the intercepted partial image in the sample image when the partial image is intercepted from the sample image. For the partial image in the same sample image, the insertion point coordinates and the interception point coordinates refer to the vertices on the same set diagonal line. For example, when the interception point coordinates select the upper left vertex and the lower right vertex, the insertion point coordinates will select the upper left vertex and the lower right vertex; when the interception point coordinates select the lower left vertex and the upper right vertex, the insertion point coordinates will select the lower left vertex and the upper right vertex.

[0064] Further, see Figure 3 , Figure 3 This is a flow chart of another embodiment of a sample image processing method of the present application. In the current embodiment, step S240 further includes steps S301 and S302, wherein the coordinates of the interception point and the insertion point of each sample image are determined based on the image size reference information and the number of objects corresponding to each sample image.

[0065] S301: Determine the coordinates of a first reference point based on the average width, the average height, and the number of objects included in each sample image.

[0066] When determining the coordinates of the interception points and the insertion points of each sample image respectively, the coordinates of the first reference point are first determined based on the average width, average height and the number of targets included in each sample image. The first reference point is the center coordinate point referenced when each partial image is inserted into the stitching reference frame, and can also be understood as the coordinates of the adjacent vertices of each partial image when stitching in the stitching reference frame. For example, when the first preset number is four, a partial image is intercepted in each sample image based on the coordinates of each interception point, and a total of four partial images are intercepted. The four intercepted partial images are then stitched in the stitching reference frame. Each of the four partial images has a vertex that is adjacent to or overlaps with a vertex of the remaining three partial images. In the technical solution provided in this application, the adjacent or overlapping vertices of the four partial images are defined as the first reference coordinate points.

[0067] Furthermore, step S301 further includes: calculating a first reference point coordinate range using the average width, average height, and the number of objects included in each sample image; and randomly determining the first reference point coordinates within the first reference point coordinate range. In the current embodiment, the first reference point coordinate range is first calculated based on the average width, average height, and the number of objects included in each sample image, and then the first reference point coordinates are determined within the determined first reference point coordinate range using a random function. The first reference point coordinate range is the area within the stitching reference frame that can be selected as the first reference point. The random function can be set based on actual needs and is not limited here.

[0068] It should be noted that in another embodiment, when the same sample image group is to be processed to obtain multiple extended images, different first reference point coordinates can be randomly selected multiple times within the determined first reference point coordinate range. Then, based on each of the different determined first reference point coordinates, different sets of interception point coordinates and insertion point coordinates corresponding to the first reference point coordinates are determined. Then, based on each set of determined interception point coordinates and insertion point coordinates, partial images are captured and stitched together to obtain multiple different extended images. For example, when the same sample image group is to be processed to obtain two extended images, and the sample image group includes four images, two first reference point coordinates Q1 and Q2 are randomly determined within the determined first reference point coordinate range. The interception point coordinates and insertion point coordinates corresponding to the four sample images are determined based on Q1. Four partial images are then captured from the four sample images based on the determined interception point coordinates. The four partial images are then inserted into the stitching reference frame based on the corresponding insertion point coordinates to obtain the extended image T1. At the same time, the interception point coordinates and insertion point coordinates corresponding to the four sample images are determined based on Q2, and then four local images are obtained from the four sample images based on the determined interception point coordinates. The four local images are then inserted into the stitching reference frame using the obtained insertion point coordinates, thereby obtaining the extended image T2. It should be noted that the first reference point coordinates Q1 and the first reference point coordinates Q2 are different coordinate points, and the interception point coordinates corresponding to the same sample image obtained based on the first reference point coordinates Q1 and the first reference point coordinates Q2 are different. Similarly, the insertion point coordinates corresponding to the same sample image obtained based on the first reference point coordinates Q1 and the first reference point coordinates Q2 are also different. Correspondingly, the local images intercepted from the same sample image based on different first reference point coordinates are different, thereby achieving different extended images based on the same sample image group.

[0069] It is understandable that in other embodiments, it is also possible to process the same sample image group to obtain other different numbers of extended images, which can be specifically configured according to actual needs.

[0070] Furthermore, please combine Figure 4 , Figure 4 Schematic diagram of an application scenario in an embodiment of a sample image processing method of the present application. In the current embodiment, the first preset number is four, that is, four images are stitched together, specifically by calculating and determining the coordinate range of the first reference point using the following formulas (1) and (2).

[0071] The formula for determining the coordinate range of the first reference point M (Xc, Yc) based on the number of objects, average width, and average height included in each sample image in the first preset number of sample images is as follows:

[0072]

[0073]

[0074] Where p1, p2, p3, and p4 are the number of targets in each of the four sample images. W is the average width of the four sample images, H is the average height of the four sample images, and U represents the average distribution. The range of the horizontal coordinate of the first reference point is determined using the above formula (1), and Xc is a randomly determined value within the range of the horizontal coordinate determined by formula (1). The range of the vertical coordinate of the first reference point is determined using the above formula (2), and Yc is a randomly determined value within the range of the vertical coordinate determined by formula (2). Figure 4 ① to ④ are used to represent the partial images intercepted from the sample image. As mentioned above, the number of targets included in the sample image corresponding to the partial images ① to ④ are represented by p1, p2, p3, and p4 respectively.

[0075] S302: Determine the coordinates of the interception points and the insertion points of each sample image using the first reference point coordinates, the average width, and the average height.

[0076] After the first reference point coordinates are determined, the interception point coordinates and the insertion point coordinates of each sample image are determined using the determined first reference point coordinates, the average width, and the average height.

[0077] In one embodiment, when the first preset number is four, that is, when four images are stitched together, the following formulas (3) to (6) are used to calculate and obtain the coordinates of the interception point and the insertion point.

[0078] Please combine Figure 4 The formulas for obtaining the coordinates of the interception points and the insertion points corresponding to each sample image based on the first reference point coordinates (Xc, Yc), the average width W of the first preset number of sample images, and the average height H of the first preset number of sample images are as follows.

[0079] The interception point coordinates (X1b, Y1b, X1b', Y1b') and insertion point coordinates (X1a, Y1a, X1a', Y1a') of the sample image corresponding to the partial image ① are calculated based on formula (3). Formula (3) is as follows:

[0080] X1a,Y1a,X1a',Y1a'=max(Xc-W,0),max(Yc-H,0),Xc,Yc;

[0081] X1b,Y1b,X1b',Y1b'=W-(X1a'-X1a),H-(Y1a'-Y1a),W,H (3)

[0082] Among them, X1a and Y1a refer to the horizontal and vertical coordinates of a vertex on the diagonal line of the partial image ① in the stitching reference frame when the partial image ① is inserted into the stitching reference frame ( Figure 4 In the illustrated embodiment, the diagonal line is set to be the diagonal line between the upper left vertex and the lower right vertex. X1a' and Y1a' refer to the horizontal and vertical coordinates of another vertex on the set diagonal line in the local image ① in the stitching reference frame. In formula (3), max(Xc-W,0) is used to indicate that the larger value between Xc-W and 0 is output as X1a, and max(Yc-H,0) is used to indicate that the larger value between Yc-H and 0 is output as Y1a. X1b and Y1b refer to the horizontal and vertical coordinates of a vertex on the set diagonal line in the local image ① when the local image ① is intercepted from the sample image. X1b' and Y1b' refer to the horizontal and vertical coordinates of another vertex on the set diagonal line in the local image ① in the sample image.

[0083] The interception point coordinates (X2b, Y2b, X2b', Y2b') and insertion point coordinates (X2a, Y2a, X2a', Y2a') of the sample image corresponding to the partial image ② are calculated based on formula (4). Formula (4) is as follows:

[0084] X2a,Y2a,X2a',Y2a'=max(Xc-W,0),Yc,Xc,min(2*H,Yc+H);

[0085] X2b,Y2b,X2b',Y2b'=W-(X2a'-X2a),0,W,min(Y2a'-Y2a,H) (4)

[0086] Where X2a and Y2a represent the horizontal and vertical coordinates of a vertex on the diagonal line of partial image ② when it is inserted into the stitching reference frame, and X2a' and Y2a' represent the horizontal and vertical coordinates of the other vertex on the diagonal line of partial image ② in the stitching reference frame. In formula (4), max(Xc-W,0) indicates that the larger value of Xc-W and 0 is output as X2a, and min(2*H,Yc+H) indicates that the smaller value of 2*H and Yc+H is output as Y2a'. X2b and Y2b represent the horizontal and vertical coordinates of a vertex on the diagonal line of partial image ② when it is intercepted from the sample image, and X2b' and Y2b' represent the horizontal and vertical coordinates of the other vertex on the diagonal line of partial image ② in the sample image. Here, min(Y2a'-Y2a) is used to indicate that the smaller one of Y2a'-Y2a and H is output as Y2b'.

[0087] The interception point coordinates (X3b, Y3b, X3b', Y3b') and insertion point coordinates (X3a, Y3a, X3a', Y3a') of the sample image corresponding to the partial image ③ are calculated based on formula (5). Formula (5) is as follows:

[0088] X3a,Y3a,X3a',Y3a'=Xc,max(Yc-H,0),min(Xc+W,2*W),Yc;

[0089] X3b,Y3b,X3b',Y3b'=0,H-(Y3a'-Y3a),min(W,X3a'-X3a),H (5)

[0090] Here, X3a and Y3a represent the horizontal and vertical coordinates of a vertex on the diagonal line of partial image ③ when it is inserted into the stitching reference frame, and X3a' and Y3a' represent the horizontal and vertical coordinates of the other vertex on the diagonal line of partial image ③ in the stitching reference frame. In formula (5), max(Yc-H,0) indicates that the larger value of Yc-H and 0 is output as Y3a, and min(Xc+W,2*W) indicates that the smaller value of Xc+W and 2*W is output as X3a'. X3b and Y3b refer to the horizontal and vertical coordinates of a vertex on the diagonal line set in local image ③ when it is intercepted from the sample image. X3b' and Y3b' refer to the horizontal and vertical coordinates of another vertex on the diagonal line set in local image ③ in the sample image. min(W,X3a'-X3a) is used to indicate that the smaller value of W or X3a'-X3a is output as X3b'.

[0091] The interception point coordinates (X4b, Y4b, X4b', Y4b') and insertion point coordinates (X4a, Y4a, X4a', Y4a') of the sample image corresponding to the partial image ④ are obtained based on the calculation of formula (6).

[0092] X4a,Y4a,X4a',Y4a'=Xc,Yc,min(Xc+W,W*2),min(H*2,Yc+H);

[0093] X4b,Y4b,X4b',Y4b'=0,0,min(W,X4a'-X4a),min(Y2a'-Y2a,H) (6)

[0094] Here, X4a and Y4a represent the horizontal and vertical coordinates of a vertex on the diagonal line of partial image ④ when it is inserted into the stitching reference frame, and X4a' and Y4a' represent the horizontal and vertical coordinates of the other vertex on the diagonal line of partial image ④ in the stitching reference frame. In formula (6), min(Xc+W,W*2) indicates that the smaller value of Xc+W and 2*W is output as X4a', and min(H*2,Yc+H) indicates that the smaller value of 2*H and Yc+H is output as Y4a'. X4b and Y4b refer to the horizontal and vertical coordinates of a vertex on the diagonal line set in local image ④ when the local image is intercepted from the sample image. X4b' and Y4b' refer to the horizontal and vertical coordinates of another vertex on the diagonal line set in local image ④ in the sample image. min(W,X4a'-X4a) is used to indicate that the smaller value of W and X4a'-X4a is output as X4b'. min(Y2a'-Y2a,H) is used to indicate that the smaller value of Y2a'-Y2a and H is output as Y4b'.

[0095] Furthermore, please combine Figure 4 and Figure 6 In the technical solution provided by the present application, due to the diversity of the sizes of the captured partial images, the stitched partial images may not be able to fill the stitching reference frame. Therefore, after stitching the partial images based on the insertion point coordinates of each sample image, the technical solution provided by the present application further includes: filling the remaining part of the stitching reference frame to obtain a stitched image, and output the stitched image. The remaining part of the stitching reference frame refers to the position that is still not filled by any partial image after the first preset number of partial images are inserted into the stitching reference frame (the remaining part of the stitching reference frame can be specifically referred to as Figure 6 There is no restriction on the filling method of the remaining part of the stitching reference frame, which is subject to the actual setting.

[0096] S250: Based on the coordinates of the interception points of the respective sample images, clipping the corresponding sample images to obtain partial images.

[0097] After determining the interception point coordinates and the insertion point coordinates of each sample image, local images are intercepted from each sample image based on the interception point coordinates corresponding to each sample image.

[0098] In the current embodiment, the above-mentioned step S140 of stitching and cropping the partial images further includes step S260.

[0099] S260: Based on the insertion point coordinates of each sample image, corresponding to each local image obtained by stitching and cropping, an extended image is obtained and output.

[0100] After capturing each partial image, the captured partial images are further inserted into the stitching reference frame according to the coordinates of their corresponding insertion points, thereby stitching together to obtain an extended image, which is then output. The stitching reference frame is a pre-set frame structure used for stitching images, and the size of the stitching reference frame is determined based on the average height and average width of a first preset number of sample images. When stitching four sample images, and the average width of these four sample images is W and the average height of these four sample images is H, the width of the stitching reference frame can be set to 2*W, and the height of the stitching reference frame can be set to 2*H.

[0101] Furthermore, since the average width and average height of the sample images stitched together each time may be different, the width and height of the stitching reference frame can be set to follow the average width and average height of the first preset number of sample images and are adjusted accordingly. There is no unique limitation on the size of the stitching reference frame.

[0102] Figure 2The technical solution provided in the corresponding embodiment obtains a first preset number of sample images from a training sample set, and determines the average width and average height of the first preset number of sample images; then, based on the average width of the first preset number of sample images, the average height of the first preset number of sample images and the number of targets included in each sample image, determines the interception point coordinates and insertion point coordinates of each sample image respectively, intercepts a local image from each corresponding sample image based on the interception point coordinates of each sample image, and then splices each local image based on the insertion point coordinates of each sample image to obtain an extended image, that is, by realizing the interception point target determined in combination with the number of targets included in the sample image, adjusting the coordinates of the interception point in each sample image using the number of targets included in the sample image, adjusting the size of the local image intercepted from each sample image, and then splicing the local images intercepted from each sample image based on the determined insertion point coordinates to obtain an extended image, thereby improving and reducing the amount of useless information generated in the sample image, and at the same time increasing the frequency of appearance of the target frame.

[0103] Please also see Figure 5 and Figure 6 , Figure 5 This is a flow chart of another embodiment of a method for processing a sample image of the present application. Figure 6 FIG. 1 is a schematic diagram of an application scenario of an embodiment of a method for processing a sample image of the present application. In the current embodiment, the method provided by the present application includes steps S501 to S506.

[0104] S501: traverse the sample training set to select a first-category image, and randomly select a second-category preset number of second-category images from the remaining sample images in the sample training set, and output the first-category image and the second-category image as a sample image group.

[0105] S502: Determine the average height and average width of the sample image group based on the size information of each sample image in the sample image group.

[0106] S503: Determine the number of targets corresponding to each sample image.

[0107] S504: Determine the coordinates of a first reference point based on the average width, the average height, and the number of objects included in each sample image.

[0108] Steps S501 to S503 are identical to steps S210 to S230 described above, and step S504 is identical to step S301 described above. For details, please refer to the above steps and their corresponding embodiments, and will not be repeated here. In the current embodiment, step S302 determines the coordinates of the interception point and the insertion point of each sample image using the first reference point coordinates, the average width, and the average height, and further includes steps S505 and S506.

[0109] S505: Perform a first displacement process in the x-axis direction and / or a second displacement process in the y-axis direction on the first reference point coordinates to obtain the second reference point coordinates.

[0110] To make the distribution of objects in each expanded image obtained after the stitching process more uniform and avoid objects being distributed in the same position in each expanded image, in the current embodiment, after determining the coordinates of the first reference point based on the average width, average height, and number of objects included in each sample image, a displacement process is further performed on the first reference coordinate point to obtain the coordinates of the second reference point. The second reference point is the point obtained by the displacement process on the first reference point.

[0111] Specifically, the obtained first reference point coordinates are subjected to a first displacement process in the x-axis direction and / or a second displacement process in the y-axis direction to obtain the second reference point coordinates. Specifically, the first and second displacement processes are performed within the coordinate system of the stitching reference frame. The width of the stitching reference frame is denoted as 2*W, and the height of the stitching reference frame is denoted as 2*H.

[0112] In one embodiment, after obtaining the coordinates of the first reference point, a first x-axis shift is performed on the first reference point to obtain the coordinates of the second reference point. In the current embodiment, no y-axis shift is performed on the first reference point; that is, the y-coordinate of the first reference point is directly output as the y-coordinate of the second reference point. The distance shifted by the first x-axis shift is determined based on the width of the stitching reference frame and the abscissa of the first reference point. The distance shifted by the first shift is 2*W - 2*Xc, and the resulting coordinates of the second reference point are (2*W - Xc, Yc).

[0113] In another embodiment, a second displacement process is performed on the obtained first reference point coordinates in the y-axis direction to obtain the second reference point coordinates. In the current embodiment, no displacement process is performed on the first reference point in the x-axis direction, that is, the abscissa of the first reference point is directly output as the abscissa of the second reference point. The distance shifted by the second displacement process is determined based on the height of the stitching reference frame and the ordinate of the first reference point. The distance shifted by the second displacement process is 2*H-2*Yc, and the coordinates of the second reference point obtained after the displacement process are (Xc, 2*H-Yc).

[0114] In yet another embodiment, the obtained first reference point coordinates are subjected to a first displacement process in the x-axis direction and a second displacement process in the y-axis direction to obtain the second reference point coordinates. That is, in the current embodiment, the first reference point is displaced in both the x-axis and y-axis directions. The first displacement process in the x-axis direction results in a displacement distance of 2*W - 2*Xc, and the second displacement process in the y-axis direction results in a displacement distance of 2*H - 2*Yc. The resulting second reference point coordinates after the displacement process are (2*W - Xc, 2*H - Yc).

[0115] Furthermore, in order to achieve a more uniform distribution of objects in an expanded image obtained by stitching together sample images in a sample training set, and thereby obtain an expanded image that can improve the accuracy of the trained model, the number of instances in which only the first displacement processing in the x-axis direction is performed on the first reference point, only the second displacement processing in the y-axis direction is performed, both the first displacement processing in the x-axis direction and the second displacement processing in the y-axis direction are performed, and no displacement processing is performed is evenly distributed according to the total number of sample images included in the training sample set (or this can also be understood as the total number of sample image groups that can be extracted from the training sample set). For example, assuming that a sample training set includes 100 sample images, and the first preset number is 4, i.e., 4 images are stitched together each time, based on the selection rule described above that each sample image group includes one image of the first category and a second preset number of images of the second category, there will be a total of 100 sample image groups. For the sample images in these 100 sample image groups, there will be 25 sample image groups whose first reference points will not be subjected to displacement processing (it can also be understood that the first reference point is subjected to displacement processing with a displacement distance of zero, that is, the first reference point coordinates are directly output as the second reference point coordinates), and the interception point coordinates and insertion point coordinates of the sample lower image are directly determined based on the first reference point coordinates, the average width and the average height; there will be 25 sample image groups whose first reference point coordinates are only subjected to the first displacement processing in the x-axis direction to obtain the second reference point coordinates; there will be 25 sample image groups whose first reference point coordinates are only subjected to the second displacement processing in the y-axis direction to obtain the second reference point coordinates; there will be 25 sample image groups whose first reference point coordinates are subjected to both the first displacement processing in the x-axis direction and the second displacement processing in the y-axis direction to obtain the second reference point coordinates.

[0116] Furthermore, in the embodiment where the first preset number is 4, it is further configured that the first reference point can be subjected to different displacements according to a preset cyclic sequence. For example, the first reference point coordinates can be subjected to displacement processing in different sample image groups in the following order: not displacing the first reference point coordinates in any direction; performing the first displacement processing only in the x-axis direction; performing the second displacement processing only in the y-axis direction; and performing the first displacement processing in the x-axis direction and the second displacement processing in the y-axis direction.

[0117] Combine Figure 6 It can be known that Figure 6 In the schematic diagram shown, Figure 6 (A) to (D) in the middle respectively show different scenarios: no displacement in any direction of the first reference point coordinates, only the first displacement processing in the x-axis direction of the first reference point coordinates, only the second displacement processing in the y-axis direction of the first reference point coordinates, and the first displacement processing in the x-axis direction and the second displacement processing in the y-axis direction of the first reference point coordinates. Figure 6 In the scenario shown in the diagram, p1 to p4 are used to represent the number of objects included in the sample images corresponding to the partial images ① to ④, respectively, and p4<p3<p2<p1. Figure 6 The shaded areas shown in (A), (C) and (D) refer to the parts of the stitching reference frame that are not covered or filled by any partial image, and can also be understood as the parts that need to be filled. Figure 6 It can be seen that when the sample images in the sample training set are stitched together, the first reference points obtained in each sample image group can be shifted in the order of Figures (A) to (D) to achieve a more uniform distribution of targets in the expanded image obtained after the stitching process, rather than obtaining an expanded image in which all targets are concentrated in a certain area, thereby obtaining an expanded image that can make model training more accurate. Figure 6 Point M in A Refers to the first reference point. Correspondingly, Xc and Yc in (A) refer to the coordinates without displacement; M B 、M C and M D Refers to the second reference point obtained by displacing the first reference point corresponding to each other, and the corresponding M B 、M C and M D The median coordinate values ​​Xc and Yc refer to the coordinates obtained after displacement.

[0118] S506: Calculating the interception point coordinates and the insertion point coordinates of each sample image based on the second reference point coordinates, the average width, and the average height.

[0119] After obtaining the coordinates of the second reference point, the coordinates of the interception point and the coordinates of the insertion point of each sample image are further calculated based on the coordinates of the second reference point, the average width, and the average height. Specifically, the coordinates of the interception point and the coordinates of the insertion point of each sample image are calculated based on the above formulas (3) to (6). It should be noted that, in the current embodiment, when the coordinates of the interception point and the coordinates of the insertion point are obtained using the above formulas (3) to (6), Xc in the formulas (3) to (6) is the horizontal coordinate of the second reference point obtained after the displacement processing, and Yc in the formulas (3) to (6) is the vertical coordinate of the second reference point obtained after the displacement processing.

[0120] S507: based on the coordinates of the interception points of the sample images, intercepting partial images from the corresponding sample images, and correspondingly stitching the partial images based on the coordinates of the insertion points of the sample images to obtain and output an extended image.

[0121] Further, please also see Figure 7 , Figure 7 FIG2 is a flow chart of another embodiment of a sample image processing method of the present application. In the current embodiment, the above step S140 stitches the cropped partial images to obtain an extended image and outputs it, further including steps S701 and S702.

[0122] S701: inserting each partial image into the stitching reference frame according to the coordinates of each insertion point to obtain a stitched image.

[0123] After obtaining the coordinates of the interception point and the insertion point, and respectively intercepting a partial image from the corresponding sample image based on the interception point coordinates of each sample image, each partial image is further inserted into the stitching reference frame according to the obtained insertion point coordinates to obtain a stitched image.

[0124] Furthermore, in one embodiment, after inserting each partial image into the stitching reference frame, if there is still an area in the stitching reference frame that is not covered by any partial image, the remaining area of ​​the stitching reference frame will be filled to obtain a stitched image. The size of the stitching reference frame is based on twice the average width and average height of the first predetermined number of sample images, that is, the width of the stitching reference frame is 2*W, and the height of the stitching reference frame is 2*H.

[0125] S702: Perform affine transformation on the spliced ​​image to obtain an extended image and output it.

[0126] To ensure that the size of the expanded image obtained by stitching better matches the size requirements of the video image and achieve better results when subsequently using the expanded image for model training or detection, after obtaining the stitched image, an affine transformation is performed on the stitched image to obtain and output the expanded image. The width of the expanded image is the average width, and the height of the expanded image is the average height.

[0127] Furthermore, when four sample images are stitched together, the following formula (7) is used to perform an affine transformation on the stitched image. Wherein, C, S, and T are respectively described in the following formulas (8) to (10). Formulas (7) to (10) are as follows:

[0128] M=C*S*T (7)

[0129]

[0130]

[0131]

[0132] It is understandable that when six or two sample images are stitched together, affine transformation parameters corresponding to the six or two sample images may be set to perform affine transformation on the stitched image.

[0133] Compared with the existing Mosaic data augmentation training image technology in YOLOv5, the expanded image obtained by the technical solution provided in the present application has a suitable aspect ratio, and in the process of generating the expanded image, it is possible to reduce the occurrence of excessive padding operations, thereby avoiding the obtained expanded image from including too much useless information. At the same time, in the technical solution provided in the present application, the coordinates of the interception points are determined according to the number of targets included in each sample image used for splicing, so that a larger local image is intercepted from the sample image including a larger number of targets, and a smaller local image is intercepted from the sample image including a smaller number of targets, thereby increasing the number of targets in the spliced ​​expanded image and increasing the frequency of occurrence of the target frame, thereby achieving better training efficiency when using the extended image provided by the present application for target detection model training.

[0134] It should be noted that, in order to achieve a better effect on sample processing, the method steps in any multiple embodiments above can be combined and superimposed without contradiction, and the specific examples are not given here one by one.

[0135] In the technical solution provided by this application, in order to avoid the imbalance of positive and negative samples in the images used for model training, which leads to a lack of positive samples involved in the calculation and affects the accuracy of the model, this application also provides a sample image processing method to overcome the above problem. Figure 8 , Figure 8 This is a flow chart of an embodiment of a method for processing a sample image of the present application.

[0136] In the current embodiment, the sample image processing method provided in this application includes steps S810 to S820.

[0137] S810: For each target frame included in each expanded image, select a first type of anchor frame corresponding to the target frame based on a size ratio between the target frame and each anchor frame.

[0138] Among them, the extended image is the image used for model training to obtain model detection, and the extended image is based on Figures 1 to 7 and the method in any corresponding embodiment thereof.

[0139] For each target frame included in each extended image, the size ratio of the target frame to the corresponding anchor frames will be determined, and then the first type of anchor frames corresponding to the target frame will be screened based on the size ratio of the target frame to the anchor frames. Among them, the target frame is a rectangular frame that the extended image has been pre-annotated based on the target. The pre-marked target frame also includes the coordinate information and width and height of the target frame. The anchor frame is a pre-set rectangular frame with different widths and heights for locating the target frame. This is specifically to find a suitable positive sample to reduce the difficulty of coordinate regression. Furthermore, the target frame and anchor frame can be set manually or automatically using a software program. The specific setting can be based on the actual setting and is not limited here.

[0140] S820: For each target frame, calculate a similarity ratio between each first-category anchor frame and the target frame, and output the first-category anchor frame whose similarity ratio is greater than or equal to a first threshold as a positive sample of the target frame.

[0141] After obtaining the first-category anchor frames for the target frames in the expanded image, the similarity ratio between each corresponding first-category anchor frame and the target frame is calculated for each target frame. The similarity ratio further includes the intersection-over-union ratio.

[0142] After obtaining the similarity ratio between each first-category anchor frame and the target frame, the first-category anchor frames whose similarity ratio is greater than or equal to a first threshold are output as positive samples of the target frame. The first threshold can be set to 0.2. It is understood that in other embodiments, the first threshold can also be set to other values, which are not listed here.

[0143] In the current embodiment, the size ratio of the target box and the anchor box is further combined with the similarity ratio between the target box and the first type of anchor box to obtain positive samples for target box screening in the extended image, thereby more accurately removing positive samples with too low quality, optimizing the detection confidence of the target, and improving the detection performance of the model.

[0144] See Figure 9 , Figure 9 FIG. 1 is a flow chart of another embodiment of a method for processing a sample image of the present application. In the current embodiment, the method provided by the present application includes steps S901 to S905.

[0145] In the current embodiment, the above step S810 filters the first type of anchor frame corresponding to each target frame included in each extended image based on the size ratio between the target frame and each anchor frame, and further includes steps S901 to S904.

[0146] S901: For each target box, determine a sample matching box of the target box.

[0147] In the current embodiment, it is first necessary to determine a sample matching frame of the current target frame for each target frame included in the expanded image, wherein the sample matching frame is determined based on the marker points of the target frame.

[0148] Furthermore, determining the sample match of the target frame in step S901 further includes: determining a second preset number of positive sample matching points based on the marker points of the target frame, and determining the sample matching frame based on the positive sample matching points. The second preset number is pre-set and is used to limit the number of positive sample matching points selected. The marker points of the target frame are coordinate points in the target frame used to mark the coordinates of the target frame, including at least the center point or the upper left vertex of the target frame. The marker points of the target frame are pre-set when performing target frame detection on the extended image.

[0149] In one embodiment, the marked point of the target frame is the coordinate point marked in the target frame, that is, with the coordinate point (x, y) marked in the target frame as the center, a second preset number of points around the coordinate point (x, y) are determined as positive sample matching points. The coordinate point marked in the target frame can be the center point of the target frame or the upper left vertex. In other embodiments, the coordinate point marked in the target frame can also be other vertices, which are not listed here one by one.

[0150] In one embodiment, the second preset number is 9, and accordingly, 8 points surrounding the coordinate point (x, y), and a total of 9 points at the coordinate point (x, y), are determined as positive sample matching points. After determining the second preset number of positive sample matching points, the largest rectangular box that can be determined by the positive sample matching points is further determined as the sample matching box.

[0151] In another embodiment, the second preset number can be set to 6, and the corresponding 5 points around the coordinate point (x, y) and the total 6 points of the coordinate point (x, y) are determined as positive sample matching points, and then the largest rectangular box that can be determined by the positive sample matching point is determined as the sample matching box.

[0152] S902: Determine an anchor frame to be verified corresponding to the target frame according to the sample matching frame of the target frame, and determine a first size ratio between the target frame and each anchor frame to be verified.

[0153] In the embodiments provided herein, anchor frames are pre-defined rectangular frames of varying widths and heights based on the points in the image. For example, each point corresponds to a certain number of anchor frames. To quickly determine a positive sample of a target frame, the current embodiment first determines the sample matching frame corresponding to the target frame. Then, based on the sample matching frame corresponding to the target frame, the anchor frame to be verified corresponding to the target frame is determined.

[0154] In one embodiment, the anchor frame to be verified may be an anchor frame whose distance from the sample matching frame is less than or equal to a preset distance. That is, in the current embodiment, the anchor frame whose distance from the sample matching frame is less than or equal to the preset distance is determined as the anchor frame to be verified, and the first size ratio between the target frame and each anchor frame to be verified is further calculated and determined.

[0155] Furthermore, in another embodiment, the anchor frame to be verified may also be an anchor frame whose distance from the marker point of the target frame is less than or equal to a preset distance.

[0156] The first size ratio is a width and / or height ratio.

[0157] For each target frame included in each expanded image, the first size ratio of the target frame and each corresponding anchor frame to be verified is calculated, that is, the width and / or height ratio of the target frame to each anchor frame to be verified is calculated.

[0158] In one embodiment, the width ratio of the target box to the anchor box to be verified is output as the first size ratio.

[0159] In another embodiment, the height ratio of the target box and the anchor box to be verified is output as the first size ratio.

[0160] In another embodiment, the width ratio and height ratio of the target frame and the anchor frame to be verified are respectively calculated, and then the average of the width ratio and the height ratio is calculated, and the average of the width ratio and the height ratio is output as the first size ratio. It is understood that in other embodiments, the sum of the width ratio and the height ratio can also be output as the first size ratio.

[0161] In another embodiment, the width ratio and height ratio of the target frame and the anchor frame to be verified are respectively calculated, and then the anchor frames to be verified whose width ratio and height ratio meet the filtering conditions are filtered according to the set width ratio filtering threshold and height ratio filtering threshold.

[0162] S903: Filter the anchor frames to be verified whose first size ratio is less than or equal to the second threshold as the second type of anchor frames.

[0163] After obtaining the first size ratio between the target box and each anchor box, anchor boxes whose first size ratio is less than or equal to a second threshold are further selected as second-category anchor boxes. In one embodiment, the second threshold can be set to 4. It is understood that since the first size ratio can include different situations, the second threshold can be adjusted based on the first size ratio, and the specific examples are not listed here.

[0164] In one embodiment, when the first size ratio is the width ratio of the target frame to the anchor frame to be verified, the anchor frames to be verified whose width ratio is less than or equal to the second threshold are screened as second-type anchor frames.

[0165] In another embodiment, when the first size ratio is the height ratio of the target frame to the anchor frame to be verified, the anchor frames to be verified whose height ratio is less than or equal to the second threshold are screened as second-type anchor frames.

[0166] In another embodiment, when the first size ratio is the average of the width ratio and the height ratio, and the average of the width ratio and the height ratio is output as the first size ratio, the anchor box to be verified of the average of the width ratio and the height ratio will be screened and determined as the second type of anchor box.

[0167] In another embodiment, when combining the height ratio and the width ratio to screen the second type of anchor frame, the anchor frame to be verified whose height ratio is less than or equal to the second threshold and whose width ratio is also less than or equal to the second threshold is regarded as the second type of anchor frame.

[0168] S904: Calculate a second size ratio between each second-category anchor frame and the sample matching frame, and determine an anchor frame whose second size ratio is less than or equal to a third threshold as a first-category anchor frame.

[0169] After determining the sample matching box, the second size ratio of each second-category anchor box to the sample matching box is further calculated, and the anchor boxes whose second size ratio is less than or equal to the point threshold are determined as first-category anchor boxes. The second size ratio includes the aspect ratio, and the third threshold can be set to 4.

[0170] Furthermore, in one embodiment, the width ratio of the second type of anchor box to the sample matching box is output as the second size ratio.

[0171] In another embodiment, the height ratio between the second type of anchor box and the sample matching box is output as the second size ratio.

[0172] In another embodiment, the width ratio and height ratio of the second type anchor box and the sample matching box are respectively calculated, and then the average of the width ratio and height ratio is calculated, and the average of the width ratio and height ratio is output as the second size ratio. It is understood that in other embodiments, the sum of the width ratio and height ratio of the second type anchor box and the sample matching box can also be output as the second size ratio. Correspondingly, the third threshold can be adjusted according to the second size ratio, which is not limited here.

[0173] S905: For each target frame, calculate the similarity ratio between each first-category anchor frame and the target frame, and output the first-category anchor frame whose similarity ratio is greater than or equal to a first threshold as a positive sample of the target frame.

[0174] Please also see Figure 8 and Figure 9 In one embodiment, the first threshold is set to 0.2, the second threshold is set to 4, the third threshold is set to 4, and the second preset number is 6. Correspondingly, for each extended image obtained, for each target frame included in the sample image, the first size ratio of the target frame to each anchor frame is determined, and the anchor frames with the first size ratio less than or equal to the second threshold (4) are screened as second-class anchor frames, and then the 6th positive sample matching point is determined according to the mark point (x, y) of the target frame, and the sample matching frame is determined based on the 6 determined positive sample matching points, and then the second size ratio of each second-class anchor frame to the current sample matching frame is calculated respectively, and the anchor frames with the second size ratio less than or equal to 4 are determined as first-class anchor frames, and finally, for each target frame, the similarity ratio of each first-class anchor frame to the target frame is calculated, and the first-class anchor frames with the similarity ratio greater than or equal to the first threshold are output as positive samples of the target frame.

[0175] See Figure 10 , Figure 10 1 is a schematic diagram of a structure of an electronic device in an embodiment of the present application. In the current embodiment, the electronic device 1000 provided by the present application includes a processor 1001 and a memory 1002 coupled to the processor 1001. The electronic device 1000 can execute Figures 1 to 9 and the method described in any corresponding embodiment.

[0176] The memory 1002 includes a local storage (not shown) and is used to store a computer program. When the computer program is executed, Figures 1 to 9 and the method described in any corresponding embodiment.

[0177] The processor 1001 is coupled to the memory 1002, and the processor 1001 is used to run the computer program to perform the above Figures 1 to 9 Furthermore, in some embodiments, the electronic device may include any one of a mobile terminal, a vehicle-mounted terminal, a camera, a computer terminal, a computer, an image acquisition device with computing and storage capabilities, a server, etc., and may also include any other device with computing and processing capabilities.

[0178] See also Figure 11 , Figure 11 This is a schematic diagram of a computer readable storage medium according to an embodiment of the present invention. The computer readable storage medium 1100 stores a computer program 1101 that can be executed by a processor. The computer program 1101 is used to implement the above Figures 1 to 9 Specifically, the computer-readable storage medium 1100 may be a memory, a personal computer, a server, a network device, or a USB flash drive, and the like, and no limitation is made here.

[0179] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing a sample image, characterized in that: The method comprises: Determine image size reference information based on size information of each sample image in the sample image group; wherein the image size reference information includes an average width and an average height of the sample image group; Determine the number of targets corresponding to each sample image respectively, where the number of targets is the number of targets contained in the corresponding sample image; Using the determined number of targets and the image size reference information, cropping each sample image respectively to obtain a local image corresponding to each sample image; Splicing and cropping the partial images to obtain an extended image and output it; Before cropping each sample image using the determined number of objects and the image size reference information, the method further includes: For each of the sample images, determining a scaling ratio of each sample image according to the width of the sample image and the average width, or the height of the sample image and the average height; Each of the sample images is scaled according to the scaling ratio.

2. The method according to claim 1, characterized in that The method of cropping each sample image by using the determined number of each target and the image size reference information to obtain a local image corresponding to each sample image includes: Determine the interception point coordinates and the insertion point coordinates of each sample image based on the image size reference information and the number of objects corresponding to each sample image; Based on the coordinates of the interception points of the sample images, respectively, the local images are obtained by cutting out the corresponding sample images; The local images obtained by stitching and cropping further include: Based on the insertion point coordinates of each of the sample images, an extended image is obtained and outputted corresponding to each of the local images obtained by stitching and cropping.

3. The method according to claim 2, characterized in that The image size reference information includes: the average width and average height of the sample image group; The determining the interception point coordinates and the insertion point coordinates of each sample image based on the image size reference information and the number of objects corresponding to each sample image respectively includes: Determine the coordinates of a first reference point based on the average width, the average height, and the number of objects included in each of the sample images; The coordinates of the interception points and the insertion points of each of the sample images are determined using the coordinates of the first reference points, the average width, and the average height.

4. The method according to claim 3, characterized in that The determining of the coordinates of the first reference point based on the average width, the average height and the number of targets included in each of the sample images further comprises: Calculate the coordinate range of the first reference point using the average width, the average height and the number of targets included in each of the sample images; The first reference point coordinates are randomly determined within the first reference point coordinate range.

5. The method according to claim 3, characterized in that: The method of determining the interception point coordinates and the insertion point coordinates of each of the sample images by using the first reference point coordinates, the average width, and the average height includes: Performing a first displacement processing in the x-axis direction and / or a second displacement processing in the y-axis direction on the first reference point coordinates to obtain a second reference point coordinates; The interception point coordinates and the insertion point coordinates of each of the sample images are calculated based on the second reference point coordinates, the average width, and the average height.

6. The method according to claim 3, characterized in that The step of obtaining and outputting an extended image based on the insertion point coordinates of each of the sample images and corresponding to each of the local images obtained by stitching and cropping includes: According to the coordinates of each insertion point, each of the partial images is inserted into the stitching reference frame to obtain a stitching picture; Perform an affine transformation on the spliced ​​image to obtain and output the extended image, where the width of the extended image is the average width and the height of the extended image is the average height.

7. The method according to claim 6, characterized in that After inserting each of the partial images into the stitching reference frame, the method further includes: The remaining part of the stitching reference frame is filled to obtain the stitching picture.

8. The method according to claim 1, characterized in that Before determining the image size reference information based on the size information of each sample image in the sample image group, the method further includes: A first-category image is selected by traversing the sample training set, and a second preset number of second-category images are randomly selected from the remaining sample images in the sample training set, and the first-category image and the second-category image are output as the sample image group.

9. A method for processing a sample image, characterized in that: The method comprises: For each target frame included in each extended image, selecting a first type of anchor frame corresponding to the target frame based on a size ratio between the target frame and each anchor frame, wherein the extended image is obtained based on the method according to any one of claims 1 to 8; For each of the target frames, a similarity ratio between each of the first-category anchor frames and the target frame is calculated, and the first-category anchor frames whose similarity ratios are greater than or equal to a first threshold are output as positive samples of the target frame.

10. The method according to claim 9, characterized in that The step of selecting, for each target frame included in each extended image, a first type of anchor frame corresponding to the target frame based on a size ratio between the target frame and each anchor frame, comprises: For each of the target frames, determining a sample matching frame of the target frame; Determine, according to the sample matching frame of the target frame, an anchor frame to be verified corresponding to the target frame, and determine a first size ratio between the target frame and each of the anchor frames to be verified, where the first size ratio is a width and / or height ratio; Screening the anchor frames to be verified whose first size ratio is less than or equal to the second threshold as the second type of anchor frames; A second size ratio between each second-category anchor box and the sample matching box is calculated, and an anchor box whose second size ratio is less than or equal to a third threshold is determined as the first-category anchor box.

11. The method according to claim 10, characterized in that The determining of the sample matching frame of the target frame includes: According to the marking points of the target frame, a second preset number of positive sample matching points are determined, and the sample matching frame is determined based on the positive sample matching points.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory coupled to the processor; wherein, The memory is used to store computer programs; The processor is configured to run the computer program to perform the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that can be executed by a processor, and the computer program is used to implement the method according to any one of claims 1 to 11.

Images