Pre-marking screening method, electronic device and storage medium
By using the reference area and area proportion related to the distance of the label object in the pre-labeling filter, the problem of inaccurate changes in the area of the labeling box in the prior art is solved, and the accuracy and reliability of the screening are improved.
Patent Information
- Application Number
- CN202411200821.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-29
AI Technical Summary
The prior art relies on the size of the area occupied by the label box in the image in pre-marking screening, resulting in inaccurate changes in the area of the label box under different environments and angles, which in turn affects the accuracy of the screening.
By obtaining candidate images in the annotation task and determining the reference area related to the distance of the annotation object to the acquisition device, the pre-notation box is used to filter and retain the pre-notation box, thereby obtaining a more accurate target image .
It improves the accuracy of pre-labeling screening, can handle labeling box changes in different environments and angles more effectively, and improves the reliability of screening results.
Smart Images

Figure CN118691782B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a pre-labeling screening method, an electronic device and a storage medium. Background Art
[0002] Image annotation has always been an important task in the field of computer vision. For example, sample images with annotated boxes are often needed for training network models such as image segmentation and object detection.
[0003] At present, due to the complexity and time-consuming nature of the annotation task, traditional manual annotation methods can no longer meet the needs of large-scale data sets. As a result, automated annotation and semi-automated annotation have gradually become research hotspots. In the prior art, incorrect annotations are screened out based on the size of the area occupied by the annotation box in the image. However, in real pre-annotation scenarios, the same annotation type annotation box will occupy different areas on the same sample image due to the distance of the environment or the angle of the camera, which will result in inaccurate screening of incorrect annotations based on the size of the area occupied by the annotation box in the image.
[0004] In view of the existing technical defects, how to provide a more accurate solution for pre-labeling screening is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention
[0005] The present application at least provides a pre-labeling screening method, an electronic device and a storage medium.
[0006] The present application provides a pre-annotation screening method, comprising: obtaining each candidate image in a labeling task, each candidate image is a sample image with a pre-annotation frame of several labeling objects, the labeling task includes at least one labeling type, and the type of each labeling object belongs to one of the at least one labeling type; determining at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeling object in each reference area from the acquisition device of the sample image; for each candidate image, retaining the pre-annotation frame in the candidate image that meets the screening condition, and obtaining a first target image corresponding to the candidate image, wherein the screening condition includes that a first area ratio of the pre-annotation frame is within a first expected range of the reference area to which the pre-annotation frame belongs, and the first area ratio is the ratio between the area of the pre-annotation frame and the area of the reference area to which the pre-annotation frame belongs.
[0007] The present application provides a pre-labeling screening device, including: an acquisition module, a determination module and a retention module; the acquisition module is used to acquire each candidate image in a labeling task, each candidate image is a sample image with a pre-labeling frame of a plurality of labeling objects, the labeling task includes at least one labeling type, and the type of each labeling object belongs to one of the at least one labeling type; the determination module is used to determine at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeling object in each reference area from the acquisition device of the sample image; the retention module is used to retain the pre-labeling frame in the candidate image that meets the screening condition for each candidate image, and obtain a first target image corresponding to the candidate image, wherein the screening condition includes that a first area ratio of the pre-labeling frame is within a first expected range of the reference area to which the pre-labeling frame belongs, and the first area ratio is the ratio between the area of the pre-labeling frame and the area of the reference area to which the pre-labeling frame belongs.
[0008] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned pre-labeling screening method.
[0009] The present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned pre-marking screening method is implemented.
[0010] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0013] Figure 1This is a schematic diagram of the process of an embodiment of the pre-marking screening method of the present application. Figure 1 ;
[0014] Figure 2 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 1 ;
[0015] Figure 3 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 2 ;
[0016] Figure 4 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 3 ;
[0017] Figure 5 This is a schematic diagram of the process of an embodiment of the pre-marking screening method of the present application. Figure 2 ;
[0018] Figure 6 It is a structural schematic diagram of an embodiment of the pre-marking screening device of the present application;
[0019] Figure 7 It is a structural schematic diagram of an embodiment of the electronic device of the present application;
[0020] Figure 8 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0021] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0022] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0023] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.
[0024] The present application provides some pre-labeling screening methods and pre-labeling screening devices. The application scenarios of the pre-labeling screening method include but are not limited to the screening of pre-labeling boxes of sample images. The executor of the pre-labeling screening method may be a pre-labeling screening device. For example, the pre-labeling screening device may be arranged in a terminal device or a server or other processing device, wherein the terminal device may be a device for pre-labeling screening, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, etc. In some possible implementations, the pre-labeling screening method may be implemented by a processor calling computer-readable instructions stored in a memory.
[0025] See also Figure 1 , Figure 1 This is a schematic diagram of the process of an embodiment of the pre-marking screening method of the present application. Figure 1 Specifically, the pre-annotation screening method may include the following steps:
[0026] Step S11: Acquire each candidate image in the labeling task.
[0027] Each candidate image is a sample image with pre-annotated frames of several annotated objects. The annotation task includes at least one annotation type, and the type of each annotated object belongs to one of the at least one annotation type.
[0028] Different annotation boxes in the candidate image correspond to different annotation objects. Different annotation boxes may correspond to the same type of annotation objects. The types of annotation objects corresponding to several pre-annotated boxes contained in the candidate image may conform to at least one annotation type of the annotation task. Several may be one or more, which will not be repeated below.
[0029] The annotation task is the setting of the annotation type required for the sample image in the current image scene. The annotation task may include at least one annotation type, and the annotation type can be set according to the screening requirements. Taking the current image scene as a traffic scene as an example, the at least one annotation type may specifically include but is not limited to: vehicles, animals, traffic lights, plants, and buildings.
[0030] The sample image is an original image without a pre-annotated frame. The pre-annotated sample image may be an initial image, and the initial image corresponding to the sample image is a sample image with pre-annotated frames of several candidate objects. Among the several candidate objects, some of the candidate objects may have types that do not belong to each type of at least one annotation type. Among them, the pre-annotated frames that do not belong to each type of at least one annotation type contained in the initial image are interference pre-annotated frames. The type of candidate objects corresponding to the interference pre-annotated frames does not belong to one of the at least one annotation type. In some application scenarios, for each sample image, from the pre-annotated frames of several candidate objects possessed by the initial image corresponding to the sample image, several pre-annotated frames of annotated objects are selected to obtain a processed initial image. The processed initial image is used as the candidate image corresponding to the sample image. For each sample image, the candidate image is a sample image with pre-annotated frames of several annotated objects. The sample image is an image that has not been detected by a preset algorithm. For each sample image, the sample image is detected by a preset algorithm to obtain the initial image corresponding to the sample image. In some application scenarios, the pre-annotated frame can be obtained by detecting the sample image by a preset algorithm. The preset algorithm may include, but is not limited to, a structured algorithm, a target recognition algorithm, etc., which are not limited here. In some application scenarios, the pre-labeling screening device may be a management terminal. The embodiments of the present application may be specifically executed by a management terminal, and the management terminal may be communicatively connected to a server, and a preset algorithm may be run on the server to pre-label the sample image on the server, and the server sends the pre-labeled sample image to the management terminal.
[0031] Exemplarily, the types of candidate objects may be vehicles, animals, traffic lights, plants, and buildings. In some application scenarios, when the labeling task includes at least one labeling type of "vehicle", the pre-labeling box of the labeling object of the labeling type "vehicle" in the candidate image may be obtained; in other application scenarios, when the labeling task includes at least one labeling type of "animal", the pre-labeling box of the labeling object of the labeling type "animal" in the candidate image may be obtained; in other application scenarios, when the labeling task includes at least one labeling type of "vehicle" and "animal", the pre-labeling box of the labeling object of the labeling type "animal" and the pre-labeling box of the labeling object of the labeling type "vehicle" in the sample image may be obtained. There may be several pre-labeling boxes for labeling objects of the same labeling type. Other situations can be deduced by analogy, and examples are not given one by one here.
[0032] Step S12: Determine at least one reference area in each candidate image.
[0033] The size of each reference area in each candidate image is related to the distance information between the labeled object in each reference area and the acquisition device of the sample image.
[0034] At least one reference area may be one or more. There is an association between the size of each annotated object in the sample image and the distance between the annotated object and the acquisition device of the sample image. In some application scenarios, the association is established in advance. The association is used to indicate that the closer the annotated object is to the acquisition device of the sample image, the larger the area occupied by the annotated object in the sample image. The pre-annotated box of each annotated object is within each reference area. The reference area is the area obtained by regional division in each candidate image. The acquisition device is an image acquisition device that can acquire the sample image corresponding to the candidate image.
[0035] In some application scenarios, according to the association relationship, the step of performing a region pre-division on each candidate image to obtain at least one reference region of each candidate image may be to obtain a desired number of qualified labeled candidate images. The qualified labeled candidate image is a first target image whose annotation quality with respect to the pre-annotated frame meets the quality qualification condition. A plurality of first target images whose annotation quality with respect to the pre-annotated frame meets the quality qualification condition constitute a qualified data set. The qualified labeled candidate image is a part or all of the images in the qualified data set. Each qualified labeled candidate image belongs to the same labeling task. The distribution of the pre-annotated frame in the expected number of qualified labeled candidate images is counted. Specifically, for each qualified labeled candidate image, the distribution of the pre-annotated frame may be the area distribution of the pre-annotated frame. Based on the area distribution, several basic regions of each qualified labeled candidate image in the same labeling task are determined. Specifically, for each basic region of each qualified labeled candidate image, the area of the pre-annotated frame in the basic region is substantially the same. For the same labeling task and the same image scene, each qualified labeled candidate image has the same number of basic regions. In addition, for each basic region, a basic expected range of the basic area ratio of the pre-annotated frame in the reference region is set. The base area ratio of the pre-annotated frame is the ratio between the area of the pre-annotated frame and the area of the base region to which the pre-annotated frame belongs. Based on the base regions of each qualified annotated candidate image, at least one base region of each candidate image belonging to the same annotation task is determined.
[0036] In some application scenarios, several basic regions are directly determined as at least one reference region of each candidate image belonging to the same annotation task. In other application scenarios, for each basic region, the size of the basic region is adjusted based on the number of pre-annotated frames in the basic region in each candidate image to obtain the reference region corresponding to the basic region. Exemplarily, if the number of pre-annotated frames in the basic region in a candidate image is 1. The basic region is determined by combining 10 pre-annotated frames of annotated objects based on the qualified annotated candidate images, that is, the basic region can accommodate the pre-annotated frames of 10 annotated objects at the same time. When the number of pre-annotated frames in the basic region in the candidate image is less than the number of pre-annotated frames that the basic region can accommodate at the same time, the size of the basic region is adjusted to obtain the reference region corresponding to the basic region.
[0037] Step S13: for each candidate image, retain the pre-annotated boxes in the candidate image that meet the screening conditions, and obtain the first target image corresponding to the candidate image.
[0038] The screening condition includes that a first area ratio of the pre-marked frame is within a first expected range of the reference region to which the pre-marked frame belongs, and the first area ratio is a ratio between an area of the pre-marked frame and an area of the reference region to which the pre-marked frame belongs.
[0039] The pre-marked frames that meet the screening conditions in each candidate image are retained to obtain the first target image corresponding to each candidate image. The screening conditions include a first screening condition, and the first screening condition includes that the first area ratio of the pre-marked frame is within the first expected range of the reference area to which the pre-marked frame belongs. That is, for each candidate image, the pre-marked frames that meet the first screening condition in the candidate image are retained to obtain the first target image corresponding to the candidate image. Specifically, for each reference area in each candidate image, the following steps are performed: for each pre-marked frame in the reference area, the first area ratio in the pre-marked frame is calculated. The annotation type corresponding to the pre-marked frame is obtained in the first expected range of the reference area to which the pre-marked frame belongs. In response to the value of the first area ratio of the pre-marked frame being within the first expected range of the reference area to which the pre-marked frame belongs, the pre-marked frame is used as the first reserved frame. If the value of the first area ratio of the pre-marked frame is not within the first expected range of the reference area to which the pre-marked frame belongs, the pre-marked frame is directly deleted. For each candidate image, the first reserved frame in each reference area is retained to obtain the first target image corresponding to the candidate image.
[0040] Because the sizes of different reference areas may be different, the first expected range of the reference area is different. The first expected range of the reference area can be a reasonable range of the first area ratio of the pre-marking box in the reference area. In some application scenarios, the management terminal sets the first expected range of the pre-marking box for the annotation object belonging to the annotation type for each annotation type in the annotation task. Exemplarily, when at least one annotation type of the annotation task includes two annotation types of "vehicle" and "animal", the pre-marking box of the annotation type belonging to "vehicle" can be set to a first expected range, and the pre-marking box of the annotation type belonging to "animal" can be set to a first expected range. It can be understood that the pre-marking box of the annotation object belonging to the same annotation type, when it is in the same reference area, the pre-marking box of the annotation object of the same annotation type uses the same first expected range. The pre-marking box of the annotation object belonging to the same annotation type, when it is in different reference areas, can use the first expected range of the reference area to which each pre-marking box belongs, because the reference areas to which each pre-marking box belongs are different. For the same annotation type, because the pre-annotated frames belong to different reference areas, the first expected ranges of the pre-annotated frames of the same annotation type may be different in different reference areas. Other situations can be deduced by analogy, and no examples are given here one by one.
[0041] In other application scenarios, the management terminal sets expected information about pre-annotation box attributes such as pre-annotation box height, width, and shape of the annotation object belonging to the annotation type for each annotation type in the annotation task. The qualified information of the pre-annotation box attributes in at least part of the image in the qualified data set is determined for each annotation type in the annotation task. For a reference area, the expected information of the pre-annotation box attributes and the qualified information of the pre-annotation box attributes are combined to determine the first expected range of the reference area to which the pre-annotation box of this annotation type belongs. By combining the preset expected information of the pre-annotation box attributes with the qualified information of the pre-annotation box attributes in the historical data, that is, the qualified information of the pre-annotation box attributes that meets the annotation quality qualification conditions in the qualified data set, the accuracy of setting the first expected range of each reference area in the candidate image can be improved.
[0042] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0043] In some embodiments, the screening condition also includes that at least one of the several second area proportions corresponding to the pre-marked frame is within the second expected range of the reference area to which the pre-marked frame belongs. The pre-marking screening method may also include the following steps: For each candidate image, the following steps are performed: First, for each pre-marked frame in the candidate image, determine the several second area proportions corresponding to the pre-marked frame. Each second area proportion is the ratio between the area of the pre-marked frame and the area of the frame to be compared. The frame to be compared is any pre-marked frame other than the pre-marked frame in the reference area to which the pre-marked frame belongs. Then, retain the pre-marked frames in the candidate image that meet the screening condition, and obtain an updated image corresponding to the candidate image. Subsequently, based on the updated image, determine the second target image.
[0044] For each candidate image, perform the following steps:
[0045] For each pre-marked frame in each reference area in the candidate image, the ratio between the area of the pre-marked frame and the area of the frame to be compared is determined as the second area ratio corresponding to the pre-marked frame. The number of the plurality of second area ratios corresponding to each pre-marked frame is one less than the total number of all pre-marked frames in the reference area. The plurality of second area ratios is one second area ratio or a plurality of second area ratios. The second expected range of the reference area to which the pre-marked frame belongs may be a reasonable range of the ratio between the area of the pre-marked frame and the area of any other pre-marked frame.
[0046] The pre-annotated boxes in each candidate image that meet the screening condition are retained to obtain an updated image corresponding to each candidate image. The screening condition includes a second screening condition, and the second screening condition includes that the screening condition also includes that at least one of the plurality of second area proportions corresponding to the pre-annotated boxes is within a second expected range of the reference area to which the pre-annotated boxes belong. That is, for each candidate image, the pre-annotated boxes in the candidate image that meet the second screening condition are retained to obtain an updated image corresponding to the candidate image.
[0047] Specifically, for each reference area in each candidate image, the following steps are performed: for each pre-labeled frame in the reference area, several second area ratios in the pre-labeled frame are calculated. In response to the fact that at least one of the several second area ratios corresponding to the pre-labeled frame has a value within the second expected range of the reference area to which the pre-labeled frame belongs, the second area ratio of the pre-labeled frame within the first expected range is determined as the target second area ratio, and the pre-labeled frame corresponding to the target second area ratio and the corresponding frame to be compared are both used as the second reserved frame. If any of the second area ratios of the several second area ratios of the pre-labeled frame is not within the first expected range of the reference area to which the pre-labeled frame belongs, the pre-labeled frame is directly deleted. For each candidate image, the second reserved frame in each reference area is retained to obtain an updated image corresponding to the candidate image.
[0048] It is understandable that, in the process of retaining the pre-annotated boxes that meet the screening conditions in each candidate image, the screening conditions include the first screening conditions and / or the second screening conditions. In some application scenarios, the screening conditions include the first screening conditions, and the above step S13 is performed. In other application scenarios, the screening conditions include the second screening conditions, and the above step of retaining the pre-annotated boxes that meet the screening conditions in the candidate images is performed to obtain the updated image corresponding to the candidate image.
[0049] In other application scenarios, when the screening conditions include the first screening condition and the second screening condition, the above step S13 and the above step of retaining the pre-annotated frame in the candidate image that meets the screening conditions to obtain the updated image corresponding to the candidate image can be executed in parallel or in series. In the case of serial execution, the execution order of the two may not be limited. Exemplarily, in some application scenarios, in the case of serial execution, it can be: first execute the above step S13, and then execute the above step of retaining the pre-annotated frame in the candidate image that meets the screening conditions to obtain the updated image corresponding to the candidate image. Specifically, after the above step S13 is executed, the above step of retaining the pre-annotated frame in the candidate image that meets the screening conditions to obtain the updated image corresponding to the candidate image includes: retaining the pre-annotated frame in the candidate image that meets the second screening condition to obtain the updated image corresponding to the candidate image. In other application scenarios, in the case of serial execution, it can be: first execute the above step of retaining the pre-annotated frame in the candidate image that meets the screening conditions to obtain the updated image corresponding to the candidate image, and then execute the above step S13. Specifically, after the step of retaining the pre-annotated boxes in the candidate images that meet the screening conditions and obtaining the updated images corresponding to the candidate images is completed, the step S13 may include: for each updated image, retaining the pre-annotated boxes in the updated images that meet the screening conditions and obtaining the first target image corresponding to the candidate image.
[0050] In some application scenarios, for each candidate image, the update image corresponding to the candidate image is directly determined as the second target image corresponding to the candidate image. In other application scenarios, for each reference area in the update image, whether to determine the update image as the second target image is determined based on the distribution of the number of pre-labeled boxes contained in the reference area. Specifically, when the distribution of the number of pre-labeled boxes contained in the reference area does not meet the expected number condition, the update image is directly determined as the second target image with unqualified labeling. When the distribution of the number of pre-labeled boxes contained in the reference area meets the expected number condition, the update image is directly determined as the second target image.
[0051] In some embodiments, the annotation task includes multiple annotation types, and the above step of determining the second target image based on the updated image may include the following steps: first, for each reference area in the updated image, obtain a first total number of pre-annotated boxes contained in the reference area and a second total number of pre-annotated boxes belonging to the same annotation type. Then, in response to the first total number in the updated image reaching a first preset value and / or each second total number reaching a second preset value of each annotation type, the updated image is determined as the second target image.
[0052] The labeling task including multiple labeling types may refer to the labeling task including at least two labeling types. The first total number may be used to represent the total number of pre-labeled boxes included in each reference area in each updated image. The second total number may be used to represent the total number of pre-labeled boxes of the same labeling type included in each reference area in each updated image.
[0053] In response to the updated image satisfying the quantity expectation condition, the updated image is determined as the second target image. The quantity expectation condition includes that the first total quantity in the updated image reaches a first preset value and / or each second total quantity in the updated image reaches a second preset value of each annotation type, and the updated image is determined as the second target image.
[0054] In some application scenarios, in response to the first total number in the updated image reaching a first preset value, the updated image is determined as the second target image. In other application scenarios, in response to the second total number in the updated image reaching a second preset value of the annotation type, the updated image is determined as the second target image. In other application scenarios, in response to the first total number in the updated image reaching the first preset value and the second total number in the updated image reaching the second preset value of the annotation type. Specifically, in the case where at least one annotation type included in the annotation task is one annotation type, the value of the first preset value is equal to the value of the second preset value. In the case where at least two annotation types are included in the annotation task, the value of the first preset value is equal to or less than the sum of the values of the second preset values.
[0055] It can be considered that further screening the updated image as the second target image based on the expected quantity condition and the number of pre-annotated boxes in the second target image reaching the expected number can help improve the accuracy of the pre-annotated boxes in the second target image and avoid removing too many pre-annotated boxes.
[0056] In some embodiments, before the above step S13, the pre-annotation screening method further includes: first, determining the positional relationship between the pre-annotated box in each candidate image and the reference area in each candidate image. Then, for each candidate image, filtering out the pre-annotated box in the candidate image that meets the selection condition, the selection condition including that at least part of the pre-annotated box is not within the reference area to which the pre-annotated box belongs.
[0057] The positional relationship is used to indicate the relationship between the position information of the pre-marked box in each candidate image and the position information of the reference area in each candidate image. The position information of the pre-marked box is used to indicate the position of the pre-marked box in the candidate image. The position information of the reference area is used to indicate the position of the reference area in the candidate image.
[0058] For each candidate image, filter out the pre-annotated boxes that meet the selection condition in the candidate image to obtain a filtered image, and use the filtered image as the candidate image to perform step S13. Specifically, filter out any pre-annotated box in the candidate image whose at least part of the area is not within the reference area to which the pre-annotated box belongs to obtain a filtered image. The above step S13 includes: for each filtered image, retain the pre-annotated boxes that meet the screening condition in the filtered image to obtain a first target image corresponding to the candidate image.
[0059] See also Figure 2 , Figure 2 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 1 .
[0060] like Figure 2 Any pre-annotated box in the candidate image shown. Set the key points that the pre-annotated box of the labeled object with the annotation type of vehicle in each reference area can contain. A total of 9 key points can be set, namely the upper left vertex, upper middle point, upper right vertex, left middle point, center point, right middle point, lower left vertex, lower middle point, and lower right vertex. Calculate the 9 key points of each pre-annotated box with the annotation type of vehicle. If the key points set in the pre-annotated box are not in the reference area, filter out the pre-annotated box of the labeled object to obtain the filtered image.
[0061] In some embodiments, the above step S12 may include the following steps: performing the following steps for each candidate image: first, obtaining the distance information between each annotated object and the acquisition device. Then, based on the pre-annotated frame of each annotated object and each distance information, the candidate image is divided into regions to obtain at least one reference region.
[0062] The distance information can be used to represent the distance information between each annotated object in each candidate image and the acquisition device.
[0063] The above-mentioned step of dividing the candidate image into regions based on the pre-annotated frames and distance information of each annotated object to obtain at least one reference region may include: for each candidate image, determining the area of the pre-annotated frames of each annotated object. According to the distance information of each annotated object, each annotated object is divided into multiple types of annotated object types according to the distance, and each annotated object type points to the distance information within a fixed range. Each annotated object type can correspond to a basic region. After determining the basic region corresponding to each annotated object type, it is determined whether the ratio between the area of the pre-annotated frame under the annotated object type and the area occupied by the basic region is greater than the threshold area. Because there may be fewer pre-annotated frames under the annotated object type, and the area occupied by the basic region is too large, so that the determined reference region is too large, so that the first area ratio of the pre-annotated frame determined by the reference region is too small, resulting in the determination that the first area ratio is invalid data, so that the first target image determined according to this first area ratio is the first target image that is unqualified for the pre-annotated frame. In response to the ratio between the area of the pre-annotated box under the annotated object type and the area occupied by the base area being greater than the threshold area, the base area corresponding to the annotated object type is directly determined as the reference area of the pre-annotated box under the annotated object type, or the base area after area fine-tuning is determined as the reference area of the pre-annotated box under the annotated object type. The base area after area fine-tuning can be obtained by the step of adjusting the size of the base area when the number of pre-annotated boxes in the base area in the candidate image is less than the number of pre-annotated boxes that can be simultaneously accommodated by the base area.
[0064] In some embodiments, before the above step S11, the pre-annotation screening method may further include the following steps: First, each initial image is obtained. Each initial image is a sample image with pre-annotated frames of several candidate objects. Then, the interfering pre-annotated frames in each initial image are deleted to obtain each candidate image. The type of the candidate object corresponding to the interfering pre-annotated frame does not belong to one of the at least one annotation type.
[0065] In some application scenarios, for each sample image, from the pre-annotated frames of several candidate objects possessed by the initial image corresponding to the sample image, select several pre-annotated frames of the annotated object, and the step of obtaining the processed initial image may be: for each initial image corresponding to the sample image, retain several pre-annotated frames of the annotated object from the pre-annotated frames of several candidate objects possessed by the initial image, obtain the processed image, and then use the processed image as the candidate image corresponding to the sample image. Specifically, from the pre-annotated frames of several candidate objects possessed by the initial image, retain the pre-annotated frame whose type of the candidate object belongs to one of the at least one annotated type, and obtain the candidate image corresponding to the sample object. Among them, the pre-annotated frame whose type of the candidate object belongs to one of the at least one annotated type among the pre-annotated frames of several candidate objects possessed by the initial image is used as the initial retained frame. For the initial image corresponding to each sample image, retain each initial retained frame to obtain the candidate image corresponding to the sample image.
[0066] In some other application scenarios, the sample image may be an image composed of several candidate objects. The sample image is processed by a preset algorithm to obtain an initial image corresponding to a pre-annotated frame having several candidate objects. Specifically, after obtaining the initial image corresponding to each sample image, the interference pre-annotated frame in each initial image is determined according to at least one annotation type in the annotation task. For any pre-annotated frame in the initial image corresponding to each sample image, if the type of the candidate object corresponding to the pre-annotated frame does not belong to one of the at least one annotation type, the pre-annotated frame is determined as an interference pre-annotated frame. For the initial image corresponding to each sample image, the interference pre-annotated frame in the initial image is deleted to obtain the candidate image corresponding to the sample image.
[0067] It can be considered that the pre-annotation boxes corresponding to all candidate objects in the initial image are screened by using at least one annotation type specified in the annotation task, and the types of pre-annotation boxes corresponding to several annotation objects in the candidate image are in accordance with at least one of the annotation types, thereby ensuring that the candidate images that can be judged by the screening conditions meet the annotation task.
[0068] In some embodiments, after the above-mentioned step S13, the pre-labeling screening method may further include the following steps: first, select at least part of the first target images to obtain a sampling atlas; then, determine whether the labeling quality of the sampling atlas with respect to the pre-labeling box meets the quality qualification conditions; subsequently, in response to the labeling quality of the sampling atlas with respect to the pre-labeling box meeting the quality qualification conditions, transmit each first target image in the labeling task to the labeling terminal.
[0069] At least some of the candidate images that have been pre-labeled and screened are selected to obtain a sampling atlas. That is, each sampling image in the sampling atlas is a candidate image that has been at least partially pre-labeled and screened. The candidate image that has been pre-labeled and screened can be the first target image, or the second target image, or the first target image and the second target image. It is understandable that the first target image can be obtained by screening the second target image according to the first screening method. The second target image can be obtained by screening the first target image according to the second screening method. It is understandable that each sampling image in the sampling atlas can be the first target image or the second target image.
[0070] In some application scenarios, the pre-annotated and screened candidate images can be randomly sampled to obtain a sampled atlas. In other application scenarios, the number of images that need to be sampled and the filtering number ranking are obtained. The filtering number ranking can be sorted by the number of pre-annotated boxes filtered out of each image in the pre-annotated and screened candidate images. The more pre-annotated boxes are filtered out in the filtering number ranking, the higher the ranking. The value corresponding to the number of images that need to be sampled is the ranking threshold. The pre-annotated and screened candidate images whose rankings in the filtering number rankings are higher than the ranking threshold are selected as the sampled images in the sampled atlas.
[0071] In some application scenarios, after obtaining the sampling atlas, the sampling atlas can be displayed on the review interface of the annotation terminal for quality inspection of each sampling image in the sampling atlas, and when confirming that the pre-annotated box in the sampling image is qualified, click the "Quality Qualified" button on the review interface, and when confirming that the pre-annotated box in the sampling image is unqualified, click the "Quality Unqualified" button on the review interface. In other application scenarios, a sidebar can also be provided on the review interface, which displays options corresponding to each sampling image in the sampling atlas in turn, so that when a certain option is selected, the sampling image corresponding to the option is displayed on the review interface. It is understandable that the management terminal sends the sampling atlas to the annotation terminal. The annotation terminal can directly send the judgment result in the above-mentioned step of judging whether the annotation quality of the pre-annotated box in the sampling atlas meets the quality qualification condition directly to the management terminal. The management terminal directly obtains the judgment result.
[0072] In some application scenarios, after obtaining the quality inspection results (i.e., qualified or unqualified) of each sampled image in the sampled atlas with respect to the pre-labeled frame, the proportion of images with unqualified pre-labeled frames in the sampled atlas can be counted as the annotation quality of the sampled atlas with respect to the pre-labeled frames. Of course, the proportion of images with qualified pre-labeled frames in the sampled atlas can also be counted as the annotation quality of the sampled atlas with respect to the pre-labeled frames, which is not limited here. Exemplarily, in the case where the annotation quality represents the former, the quality qualification condition can be set to the proportion of images being lower than the first threshold; conversely, in the case where the annotation quality represents the latter, the quality qualification condition can be set to the proportion of images being higher than the second threshold. It should be noted that the first threshold and the second threshold can be set according to the actual application needs. For example, in the case where the pre-labeling screening is relatively strict, the first threshold can be set appropriately lower, and the second threshold can be set appropriately higher.
[0073] In some application scenarios, there may be multiple annotation terminals. In this case, the candidate images pre-annotated and screened in the annotation task can be divided into subsets with the same number of annotation terminals, so that each subset can be sent to the corresponding annotation terminal for formal annotation of different sampled images in the annotation task based on the existing pre-annotated boxes. Alternatively, the candidate images pre-annotated and screened in the annotation task may not be divided, but all the candidate images pre-annotated and screened in the annotation task may be sent directly to different annotation terminals, which is not limited here.
[0074] In some embodiments, the pre-annotation screening method may further include the following steps:
[0075] First, in response to the fact that the annotation quality of the pre-annotated frame in the sampling atlas does not meet the quality qualification condition, an analysis atlas is obtained, wherein the analysis atlas includes at least one image to be analyzed in the sampling atlas, and each image to be analyzed is a first target image in the analysis atlas whose pre-annotated frame is unqualified. Then, according to the number information of the pre-annotated frames filtered out in each image to be analyzed, an analysis result is determined. Subsequently, based on the analysis result, the step of obtaining each candidate image in the annotation task is re-executed, or the step of determining at least one reference area in each candidate image is re-executed.
[0076] For the method of determining whether the sampled image in the sampled atlas is qualified with respect to the pre-annotated frame, please refer to the aforementioned content and will not be described in detail here.
[0077] At least part of the sampled images in the sampled atlas is determined as at least one image to be analyzed, and at least one image to be analyzed constitutes the analysis atlas. Wherein. Each image to be analyzed is a candidate image that has been determined to be unqualified in labeling and has been pre-labeled and screened. That is, each image to be analyzed can be a sampled image that has been determined to be unqualified in labeling, and then each image to be analyzed can be a first target image that is unqualified in labeling, or a second target image that is unqualified in labeling. The number information of the pre-labeled frames filtered out in the image to be analyzed can be the difference between the total number of pre-labeled frames contained in the initial image or candidate image corresponding to each image to be analyzed and the total number of pre-labeled frames contained in each image to be analyzed. According to the number information of the pre-labeled frames filtered out in each image to be analyzed, determining the analysis result can be determining the correlation according to the number information of the pre-labeled frames filtered out in each image to be analyzed. The correlation is the numerical relationship between the first number of pre-labeled frames filtered out of each image to be analyzed in the atlas to be analyzed and the second number of pre-labeled frames filtered out of the number in the sampled atlas or the atlas to be analyzed. The analysis results are that the correlation is positive or negative.
[0078] In some application scenarios, as mentioned above, the pre-labeled frame can be obtained by detecting the sample image by a preset algorithm. In response to the positive correlation, the preset algorithm can be optimized and the above step S11 can be re-executed. It should be noted that, in the case of a positive correlation, the more the first number of pre-labeled frames filtered out in the image to be analyzed, the more the second number of images to be analyzed from which the pre-labeled frames of the number are filtered out. Conversely, the fewer the first number of pre-labeled frames filtered out in the image to be analyzed, the fewer the second number of images to be analyzed from which the pre-labeled frames of the number are filtered out. As a special case that may exist in an actual application, for one pre-labeled frame filtered out, there is a total of one image to be analyzed in this case, for two pre-labeled frames filtered out, there are a total of two images to be analyzed in this case, and for three pre-labeled frames filtered out, there are a total of three images to be analyzed in this case. It can be seen that the correlation between the first number and the second number is positively correlated. It should be noted that, in the above example, for one pre-labeled frame to be filtered out, there is a total of one image to be analyzed in this case. When the first number is 1 in the atlas to be analyzed, that is, when the first number of pre-labeled frames to be filtered out is 1, there is a total of a second number of images to be analyzed in the atlas to be analyzed that meet the situation of filtering out the first number of pre-labeled frames of 1. At this time, the second number of images to be analyzed can indicate that there is a total of one image to be analyzed in this case. It can be understood that, in this example, when the first number of pre-labeled frames to be filtered out in the image to be analyzed is 1, the second number of images to be analyzed in the atlas to be analyzed that meet the situation of filtering out the first number of pre-labeled frames is 1. Of course, the above example is only a special case that may exist in actual applications, and does not limit the "positive correlation" situation. In this case, there is a greater possibility that there is a problem with the preset algorithm. Of course, there is also a certain probability that the first expected range and / or the second expected range are set incorrectly. In order to improve the accuracy of the pre-labeled data sent to the labeling terminal, the preset algorithm can be optimized and the above step S11 can be re-executed. In addition, the specific process of optimizing the preset algorithm can refer to technical details such as structured algorithms and target recognition algorithms, which will not be repeated here. In the above method, when the analysis result shows that the correlation is positively correlated, step S11 is re-executed, so it helps to improve the accuracy of the first target image and / or the second target image sent to the annotation terminal.
[0079] In some application scenarios, as mentioned above, when the correlation is positive, there is a high possibility that there is a problem with the preset algorithm. Therefore, in order to improve the efficiency of pre-labeling screening, in the process of re-executing the aforementioned step S11, the latest first expected range and / or second expected range of the benchmark area to which the pre-labeling box corresponding to various labeled objects belongs can be reused. During the re-execution process, there is no need to reset the benchmark area and the first expected range and / or the second expected range, that is, there is no need to repeat step S12, which helps to improve the efficiency of pre-labeling screening.
[0080] In some application scenarios, different from the aforementioned implementation, as mentioned above, when the correlation is positive, there is also a certain probability that the first expected range and / or the second expected range are set incorrectly. Therefore, in order to improve the accuracy of pre-labeling screening, it is necessary to repeat step S12 to reset the reference area and the first expected range and / or the second expected range.
[0081] In some application scenarios, in response to the correlation being negatively correlated, step S12 can be re-executed. It should be noted that, in the case of a negative correlation, the more the first number of pre-labeled frames filtered out in the image to be analyzed, the fewer the second number of images to be analyzed from which the pre-labeled frames are filtered out, and vice versa, the fewer the first number of pre-labeled frames filtered out in the image to be analyzed, the more the second number of images to be analyzed from which the pre-labeled frames are filtered out. As a special case that may exist in an actual application, for 1 pre-labeled frame filtered out, there are a total of 5 images to be analyzed in this case, for 2 pre-labeled frames filtered out, there are a total of 4 images to be analyzed in this case, and for 3 pre-labeled frames filtered out, there are a total of 2 images to be analyzed in this case. It can be seen that the correlation between the first number and the second number is negatively correlated. Of course, the above example is only a special case that may exist in an actual application, and does not limit the "negative correlation" case. In this case, it is highly likely that the reference area and the first expected range and / or the second expected range are incorrectly set. In order to improve the accuracy of the first target image and / or the second target image sent to the annotation terminal, the reference area and the first expected range and / or the second expected range may be reset. That is, re-execution of step S12 is helpful to improve the accuracy of the first target image and / or the second target image sent to the annotation terminal.
[0082] In some application scenarios, as mentioned above, the pre-labeled frame is obtained by detecting the sample image by a preset algorithm. In response to the correlation including positive correlation and negative correlation, as mentioned above, there is a high possibility that there are problems with the preset algorithm and the reference area and the first expected range and / or the second expected range. In this case, the preset algorithm can be optimized, and each step can be re-executed starting from step S11. The quality of the annotation of the pre-labeled frame in the sampled atlas meets the quality qualification condition, and each first target image and / or each second target image in the annotation task is transmitted to the annotation terminal, which helps to improve the accuracy of the first target image and / or the second target image sent to the annotation terminal.
[0083] See also Figure 3 , Figure 3 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 2 .
[0084] Taking the current image scene as a traffic scene as an example, when the labeling task contains at least one labeling type of "vehicle", the pre-labeled box of the labeling object of the labeling type of "vehicle" in the candidate image can be obtained. Figure 3 As shown, the annotated objects in the candidate image that are of the annotation type "vehicle" include a first annotated object, a second annotated object, a third annotated object, and a fourth annotated object, and the areas of each annotated object shown respectively are pre-annotated boxes corresponding to each annotated object. The first expected range of the first reference area is that the value of the first area ratio is 0.1 to 0.3. The first expected range of the second reference area is that the value of the first area ratio is 0.2 to 0.5. In some application scenarios, after the above steps of filtering out the pre-annotated boxes that meet the selection conditions in the candidate image for each candidate image, the pre-annotated box of the fourth annotated object will be filtered out, that is, the candidate image may not include the pre-annotated boxes as shown in the figure. Figure 3 The pre-annotated frame of the fourth annotated object is shown.
[0085] See also Figure 4 , Figure 4 This is a schematic diagram of a pre-annotated box in a candidate image in an embodiment of the pre-annotated screening method of the present application. Figure 3 .
[0086] Taking the current image scene as a park scene as an example, when the labeling task includes at least one labeling type of "plant" and "animal", the pre-labeling box of the labeling object with the labeling type of "plant" in the candidate image can be obtained, and the pre-labeling box of the labeling object with the labeling type of "animal" in the candidate image can be obtained. Figure 4As shown, the labeled objects of the labeling type "plant" in the candidate image include the fifth labeling object, the sixth labeling object, the eighth labeling object and the ninth labeling object, and the areas of each labeling object shown respectively are the pre-labeling boxes corresponding to each labeling object. The labeled objects of the labeling type "animal" in the candidate image include the seventh labeling object and the tenth labeling object, and the areas of each labeling object shown respectively are the pre-labeling boxes corresponding to each labeling object. The second expected range of the third reference area is that the value of the second area ratio is 0.2 to 0.8. In some application scenarios, after the above-mentioned step of determining the second target image based on the updated image, when the pre-labeling box corresponding to the tenth labeling object and the frame to be compared are the pre-labeling boxes corresponding to the ninth labeling object, the second area ratio of the pre-labeling box corresponding to the tenth labeling object determined meets the second expected range of the third reference area, that is, the second reserved boxes in the updated image are only the pre-labeling boxes corresponding to the ninth labeling object and the pre-labeling boxes corresponding to the tenth labeling object. Further, by judging whether the first total number in the updated image reaches the first preset value and / or whether each second total number in the updated image reaches the second preset value of each labeling type, the updated image is determined as the second target image. The first total number is the total number obtained by summing the pre-annotation box corresponding to the ninth annotated object and the pre-annotation box corresponding to the tenth annotated object. The second total number corresponding to the annotation type of "plant" is the total number obtained by summing the pre-annotation box corresponding to the ninth annotated object. The second total number corresponding to the annotation type of "animal" is the total number obtained by summing the pre-annotation box corresponding to the tenth annotated object. It can be understood that the more the number of pre-annotation boxes corresponding to the second target image, the higher the pre-annotation screening accuracy.
[0087] See also Figure 5 , Figure 5 This is a schematic diagram of the process of an embodiment of the pre-marking screening method of the present application. Figure 2 .
[0088] Step S50: Determine the sampling atlas. Specifically, select at least part of the candidate images that have been pre-annotated and screened to obtain the sampling atlas. The candidate images that have been pre-annotated and screened may be the first target image, the second target image, or the first target image and the second target image. It can be understood that the first target image may be obtained by screening the second target image according to the first screening method. The second target image may be obtained by screening the first target image according to the second screening method. Step S51: Determine whether the annotation quality of the sampling atlas with respect to the pre-annotated frame meets the quality qualification condition. If the judgment result is yes, execute step S52. Step S52: In response to the annotation quality of the sampling atlas with respect to the pre-annotated frame meeting the quality qualification condition, transmit each first target image in the annotation task to the annotation terminal. Specifically, in response to the annotation quality of the sampling atlas with respect to the pre-annotated frame meeting the quality qualification condition, transmit each first target image and / or each second target image in the annotation task to the annotation terminal. If the judgment result is no, execute steps S53, S54 and S55 in sequence. Step S53: In response to the quality of the annotation of the pre-annotated frame in the sampled atlas not meeting the quality qualification condition, the analysis atlas is obtained. Step S54: According to the number of pre-annotated frames filtered out in each image to be analyzed, the analysis result is determined. Step S55: Based on the analysis result, the step of obtaining each candidate image in the annotation task is re-executed, or the step of determining at least one reference area in each candidate image is re-executed. Steps S53, S54 and S55 can refer to the above content and will not be repeated here.
[0089] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0090] See also Figure 6 , Figure 6It is a structural schematic diagram of an embodiment of the pre-annotation screening device of the present application. The pre-annotation screening device 60 includes an acquisition module 61, a determination module 62 and a retention module 63; the acquisition module 61 is used to acquire each candidate image in the annotation task, each candidate image is a sample image with a pre-annotated frame of several annotated objects, the annotation task includes at least one annotation type, and the type of each annotated object belongs to one of the at least one annotation type; the determination module 62 is used to determine at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the annotated object in each reference area from the acquisition device of the sample image; the retention module 63 is used to retain the pre-annotated frame in the candidate image that meets the screening condition for each candidate image, and obtain the first target image corresponding to the candidate image, the screening condition includes that the first area ratio of the pre-annotated frame is within the first expected range of the reference area to which the pre-annotated frame belongs, and the first area ratio is the ratio between the area of the pre-annotated frame and the area of the reference area to which the pre-annotated frame belongs.
[0091] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0092] Please refer to the pre-annotation screening method for the functions performed by each module, which will not be repeated here.
[0093] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 70 includes a memory 71 and a processor 72, and the processor 72 is used to execute program instructions stored in the memory 71 to implement the steps in the above-mentioned pre-labeling screening method embodiment. In a specific implementation scenario, the electronic device 70 may include but is not limited to: a multi-camera device, a microcomputer, and a server. In addition, the electronic device 70 may also include a laptop computer, a tablet computer and other mobile devices, which are not limited here.
[0094] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in the above-mentioned pre-labeling screening method embodiment. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by an integrated circuit chip.
[0095] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0096] See also Figure 8 , Figure 8 The computer-readable storage medium 80 stores program instructions 801, which implement the steps in any of the above-mentioned pre-marking screening method embodiments when executed by a processor.
[0097] The above scheme obtains each candidate image in the labeling task, each candidate image is a sample image with a pre-labeled frame of several labeled objects, and determines at least one reference area in each candidate image, wherein the size of each reference area in each candidate image is related to the distance information of the labeled object in each reference area from the acquisition device of the sample image, and the first area ratio is the ratio between the area of the pre-labeled frame and the area of the reference area to which the pre-labeled frame belongs. The pre-labeled frame in each candidate image is retained by using the first area ratio related to the reference area in each candidate image to obtain the first target image corresponding to each candidate image. Compared with the prior art that only relies on the size of the area occupied by the pre-labeled frame in the entire image to screen the pre-labeled frame, the present application can screen the pre-labeled frame by the reference area and the first area ratio, thereby improving the accuracy of pre-labeling screening.
[0098] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0099] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0100] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0102] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
Claims
1. A pre-marking screening method, characterized in that: The method comprises: Acquire each candidate image in a labeling task, each candidate image being a sample image with a pre-labeled frame of a plurality of labeling objects, the labeling task comprising at least one labeling type, and the type of each labeling object belonging to one of the at least one labeling type; Determine at least one reference area in each of the candidate images, wherein the size of each of the reference areas in each of the candidate images is related to distance information of the annotated object in each of the reference areas from the acquisition device of the sample image; For each of the candidate images, retaining a pre-annotated frame in the candidate image that meets a screening condition, and obtaining a first target image corresponding to the candidate image, wherein the screening condition includes that a first area ratio of the pre-annotated frame is within a first expected range of a reference region to which the pre-annotated frame belongs, and the first area ratio is a ratio between an area of the pre-annotated frame and an area of the reference region to which the pre-annotated frame belongs; The determining of at least one reference area in each of the candidate images comprises: For each candidate image, the following steps are performed: Acquiring distance information between each of the marked objects and the acquisition device; Based on the pre-annotated boxes of the annotated objects and the distance information, the candidate image is divided into regions to obtain the at least one reference region.
2. The method according to claim 1, characterized in that The screening condition also includes that at least one of the plurality of second area proportions corresponding to the pre-marked frame is within a second expected range of the reference area to which the pre-marked frame belongs, and the method further includes: For each candidate image, perform the following steps: For each of the pre-marked frames in the candidate image, determining a plurality of second area proportions corresponding to the pre-marked frame, each of the second area proportions being a ratio between an area of the pre-marked frame and an area of a frame to be compared, the frame to be compared being any pre-marked frame other than the pre-marked frame in a reference area to which the pre-marked frame belongs; retaining the pre-annotated frames in the candidate image that meet the screening condition, and obtaining an updated image corresponding to the candidate image; Based on the update image, a second target image is determined.
3. The method according to claim 2, characterized in that The annotation task includes a plurality of annotation types, and determining a second target image based on the update image includes: For each of the reference areas in the updated image, obtaining a first total number of pre-annotated boxes contained in the reference area and a second total number of pre-annotated boxes belonging to the same annotation type; In response to the first total number in the updated image reaching a first preset value and / or each of the second total numbers reaching a second preset value of each of the annotation types, the updated image is determined as the second target image.
4. The method according to any one of claims 1 to 3, characterized in that Before retaining the pre-annotated frame that meets the screening condition in each candidate image and obtaining the first target image corresponding to the candidate image, the method further includes: Determining a positional relationship between a pre-annotated frame in each of the candidate images and a reference area in each of the candidate images; For each of the candidate images, pre-marked boxes in the candidate images that meet a selection condition are filtered out, where the selection condition includes that at least a portion of the pre-marked box is not within a reference area to which the pre-marked box belongs.
5. The method according to any one of claims 1 to 3, characterized in that: Before acquiring each candidate image in the labeling task, the method further includes: Acquire each initial image, each of the initial images being a sample image with pre-labeled frames of a plurality of candidate objects; The interfering pre-marking boxes in each of the initial images are deleted to obtain each of the candidate images, and the type of the candidate object corresponding to the interfering pre-marking box does not belong to one of the at least one marking type.
6. The method according to any one of claims 1 to 3, characterized in that: After retaining the pre-annotated frame that meets the screening condition in each candidate image and obtaining the first target image corresponding to the candidate image, the method further includes: Select at least part of the first target image to obtain a sampling atlas; Determining whether the quality of the annotation of the pre-annotated frame in the sampled atlas meets the quality qualification condition; In response to the fact that the annotation quality of the sampled atlas with respect to the pre-annotated frame meets a qualified quality condition, each of the first target images in the annotation task is transmitted to the annotation terminal.
7. The method according to claim 6, characterized in that The method further comprises: In response to the quality of the annotation of the pre-annotated frame in the sampling atlas not satisfying the qualified quality condition, acquiring an analysis atlas, wherein the analysis atlas includes at least one image to be analyzed in the sampling atlas, each of the images to be analyzed being a first target image in the analysis atlas whose pre-annotated frame is unqualified; Determining an analysis result according to information on the number of pre-marked frames filtered out in each of the images to be analyzed; Based on the analysis result, the step of acquiring each candidate image in the labeling task is re-executed, or the step of determining at least one reference area in each candidate image is re-executed.
8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 7.
9. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Sample labeling method and computer storage medium
CN110135426A
Single turnout state detection system and detection method and semi-automatic data labeling method
CN115601558A
